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University  (DAU)  for  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA) 
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needed  by  naval  acquisition  systems  engineers  are  addressed  in  the  SPRDE  training 
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EXECUTIVE  SUMMARY 


This  research  examines  the  extent  to  which  the  Defense  Acquisition  University  DAU 
Systems  Planning,  Research,  Development,  and  Engineering  (SPRDE)  Systems 
Engineering  (SE)  training  requirements  for  Defense  Acquisition  Workforce  Improvement 
Act  (DAWIA)  certification  accurately  reflect  the  competencies  needed  to  perform  as  a 
proficient  systems  engineer  in  the  naval  acquisition  enterprise.  Similarities  and 
differences  between  the  DAU  SPRDE-SE  training  curriculum  and  the  KSAs  defined  by 
the  Naval  Postgraduate  School  (NPS)  SE  Competency  Model  are  analyzed  and  the 
corresponding  gaps  and  overlaps  in  training  are  identified.  As  secondary  objectives,  the 
DAU  SPRDE-SE  training  curriculum  is  further  analyzed  and  compared  to  the  DAU 
SPRDE  Competency  Model  to  determine  the  strength  and  consistency  of  the  program 
itself.  Finally,  this  research  provides  conclusions  and  recommendations  as  well  as  areas 
further  research. 

This  research  analyzed  data  drawn  directly  from  the  DAU  iCatalog  and  compared 
and  contrasted  it  with  the  knowledge,  skills  and  abilities  (KSAs)  required  to  perform  as  a 
proficient  naval  acquisition  systems  engineer,  as  identified  by  the  NPS  SE  Competency 
Model.  It  was  found  that  substantial  gaps  exist  in  the  training  of  this  highly  specialized 
workforce. 

First,  the  cognitive  and  affective  levels  of  the  SPRDE-SE  curriculum  were 
compared  to  those  of  the  KSAs  in  the  NPS  model.  A  considerable  gap  or  lack  of 
development  of  skills  was  found  pertaining  to  the  ability  to  apply  knowledge  and  as  well 
as  all  skills  falling  into  the  affective  domain.  These  affective  skills  include  abilities  such 
as  judgment,  negotiation,  and  value  which  are  crucial  to  the  successful  performance  of 
systems  engineers  and  are  part  of  what  separates  systems  engineers  from  other  engineers. 

Secondly,  the  DAU  SPRDE-SE  course  and  learning  objectives  were  directly 
compared  to  the  KSAs  identified  in  the  NPS  SE  Competency  Model  to  detennine  more 
specific  training  gaps.  It  was  found  that  a  significant  majority  (73  percent)  of  these  KSAs 
are  not  addressed  by  the  DAU  SPRDE-SE  training  curriculum. 


xv 


To  address  the  secondary  research  objectives,  the  cognitive  levels  of  the  SPRDE- 
SE  Enabling  Learning  Objectives  (ELOs)  were  compared  to  those  of  the  overarching 
Course  Learning/Performance  Objectives  (CL/POs)  to  detennine  whether  the  course 
learning  objectives  were  supported  by  the  curriculum.  It  was  found  that  over  half  of  the 
overarching  CL/POs  are  comprised  of  inconsistent  ELOs  and  that  at  least  12  percent  of 
the  CL/POs  are  not  sufficiently  supported  by  the  underlying  ELOs. 

The  DAU  SPRDE-SE  curriculum  was  then  mapped  to  its  own  competency  model 
to  establish  the  level  of  strength  and  consistency  in  the  DAU  SPRDE  program  itself.  It 
was  found  that  five  competencies  in  the  DAU  SPRDE  Competency  Model  are  not 
addressed  in  any  way  by  the  DAU  SPRDE-SE  curriculum  and  that  18  percent  of  the 
SPRDE-SE  course  objectives  apply  to  competencies  not  included  in  the  SPRDE  model. 
This  suggests  that  further  research  into  the  effectiveness  and  validity  of  the  SPRDE-SE 
curriculum  is  warranted.  Curriculum  should  be  amended  to  include  the  competencies  not 
currently  addressed  and  additional  competencies  which  are  addressed  by  the  curriculum 
should  be  added  to  the  competency  model. 

Another  important  finding  is  that  the  DAU  SPRDE-SE  training  curriculum  is 
robust  with  regard  to  acquisition  training.  Approximately  a  third  of  the  curriculum 
focuses  on  this  essential  competency.  This  is  an  important  and  highly  desirable  finding 
and  helps  to  distinguish  the  DAU  SPRDE  Competency  Model  from  other  more  generic 
systems  engineering  competency  models. 

The  overarching  problem  addressed  by  this  research  is  whether  or  not  Systems 
Engineering  competencies  are  being  sufficiently  developed  by  the  SE  workforce.  Upper 
level  concern  exists  that  current  training  requirements  for  Systems  Engineers  are  not 
sufficient  to  produce  and  support  the  high  level  SE  workforce  required  for  successful 
acquisition  programs.  This  research  has  established  that,  while  the  DAU  SPRDE-SE 
program  provides  DAWIA  certification  and  some  of  the  necessary  training,  it  does  not 
provide  comprehensive  proficiency  training  for  this  crucial  sector  of  the  acquisition 
workforce. 
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How  and  where  higher  cognitive  and  affective  abilities  are  being  developed  needs 
to  be  determined  as  well  as  sources  of  training  for  the  KSAs  not  addressed  in  DAU 
SPRDE  certification  program.  These  sources  may  include  formal  education,  work 
experience  and  others.  Furthermore,  this  research  recommends  a  closer  analysis  of  the 
DAU  SPRDE  program  to  insure  that  the  associated  educational  objectives  and  outcomes 
are  being  achieved.  DAU  SPRDE-Se  training  is  an  important  component  of  the  overall 
competencies  developed  by  the  Naval  systems  engineering  workforce.  The  validity  and 
effectiveness  of  this  training  must  be  assured. 
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I.  INTRODUCTION 


Workforce  size  is  important  but  quality  is  paramount. 

Ashton  B.  Carter  (2010) 

This  chapter  will  provide  a  history  of  Department  of  Defense  (DoD)  acquisition 
workforce  issues,  the  Defense  Acquisition  University  and  the  Systems  Planning, 
Research,  Development,  and  Engineering  (SPRDE)  career  fields.  The  evolution  of  both 
the  SPRDE  and  Naval  Postgraduate  School  (NPS)  Systems  Engineering  (SE) 
Competency  Models  will  be  examined.  This  background  information  will  reveal  inherent 
problems  in  SE  workforce  training  which  will  ultimately  be  addressed  in  this  research.  A 
broad  problem  statement  will  lead  to  research  objectives  and  finally  develop  into  specific 
research  questions.  This  chapter  will  also  address  the  importance  of  this  research  and 
methodology  by  which  it  will  be  conducted. 

A.  BACKGROUND 

1.  DAWIA  and  DAU 

For  over  a  half  century,  the  U.S.  Government  has  struggled  to  define  and 
implement  the  training  and  education  necessary  to  produce  qualified,  effective 
acquisition  professionals.  The  post-war  era  brought  an  onslaught  of  heightened 
technology  and  complexity  to  the  government  acquisition  arena  and  numerous  studies 
were  commissioned  by  Congress  and  the  president  to  address  deficiencies  in  the 
acquisition  workforce;  many  of  which  have  influenced  our  current  acquisition  workforce 
certification  structure  today.  As  early  as  1955,  the  Hoover  Commission  highlighted  the 
need  for  identification  of  procurement  career  paths  and  the  1972  Commission  on 
Government  Procurement  reported  that  a  university  structure  was  required  to  further 
acquisition  career  management. 

The  1980s,  however,  brought  increasing  acquisition  scandal  and  Congress  and  the 
President  were  further  compelled  by  public  acrimony  to  produce  meaningful 
improvements  to  the  acquisition  process.  One  outcome  of  these  efforts  was  the 
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establishment  of  the  Packard  Commission.  Lead  by  former  Deputy  Secretary  of  Defense 
David  Packard,  the  Commission  reported  in  1986  that  the  Department  of  Defense  (DoD) 
acquisition  workforce  (AWF)  was  undertrained,  inexperienced  and  underpaid  and  their 
recommendations  centered  on  the  creation  of  a  group  of  highly-trained,  highly-skilled 
acquisition  professionals.  This  counsel  became  the  focus  of  the  1990  legislation  that 
guides  acquisition  workforce  development  today  (Layton,  2007,  pp.  4-5). 

Two  other  concurrent  studies  performed  in  the  late  1980s  had  an  important 
bearing  on  AWF  career  development;  the  Defense  Management  Report,  conducted  by  the 
DoD,  and  the  Quality  and  Professionalism  of  the  Acquisition  Workforce,  conducted  by 
the  House  Armed  Services  Committee.  John  Betti,  then  Undersecretary  of  Defense  for 
Acquisition  (USD  (A),  felt  that  previous  reforms  were  unsuccessful  because  they  were 
mandated  by  Congress  rather  than  developed  by  the  agencies  themselves.  Congress’  issue 
with  the  resultant  recommendations  from  the  DoD  was  that  they  did  not  promote  a 
unified  approach;  rather  the  DoD  proposed  training  and  education  significantly  different 
for  each  service.  The  concurrent  House  Armed  Services  Committee’s  report  concluded 
that  acquisition  required  professional  skills  and  that,  in  addition  to  lack  of  college 
education,  high  turnover,  and  the  underutilization  of  civilians,  the  inconsistent  training 
and  education  programs  of  the  various  DoD  agencies  were  causing  major  gaps  in  AWF 
career  development.  While  the  DoD  balked  at  the  implications  of  additional  legislation, 
Congress  was  detennined  to  implement  statute-based  acquisition  training  reform 
resulting  in  the  Defense  Acquisition  Workforce  Improvement  Act. 

The  Defense  Acquisition  Workforce  Improvement  Act  (Title  10  U.S.C.  1701- 
1764),  also  known  as  DAWIA,  was  introduced  by  Congressmen  Nicholas  Mavroules,  a 
member  of  House  Armed  Services  Committee,  and  was  signed  into  law  as  a  part  of  the 
1991  National  Defense  Appropriation  Act  (NDAA)  in  November  1990.  According  to 
Mavroules,  “Part  of  our  intention  in  passing  this  legislation  was  to  ensure  that  the  sound, 
common-sense  recommendations  made  by  those  numerous  commissions  are 
implemented”  (Mavroules,  1991,  p.  18). 

DAWIA  requires  the  DoD,  acting  through  the  Secretary  of  Defense  (SecDef)  and 

the  Under  Secretary  of  Defense  (USD)  for  Acquisition,  Technology  and  Logistics 
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(AT&L),  to  establish  minimum  education,  training  and  experience  requirements;  to 
establish  and  maintain  formal  career  paths  for  both  military  and  civilian  Acquisition 
Workforce  members;  to  implement  a  formal  certification  process  and  to  establish  an 
acquisition  corps  of  highly  qualified  acquisition  professionals.  It  also  mandates  that  a 
Defense  Acquisition  University  is  established  and  maintained  to  provide  for  “(1)  the 
professional  educational  development  and  training  of  the  acquisition  workforce;  and  (2) 
research  and  analysis  of  defense  acquisition  policy  issues  from  an  academic  perspective” 
(Title  10  U.S.C.  146).  Finally,  DAWIA  also  requires  a  collaborative,  uniform  approach 
between  services.  The  overarching  goal  is  to  establish  a  professional,  motivated, 
educated,  highly-skilled,  highly-  trained  acquisition  workforce. 

In  1991,  the  DoD  Manual  5000. 52-M,  Career  Development  Program  for 
Acquisition  Personnel  implemented  DAWIA  creating  the  Defense  Acquisition  University 
(DAU).  The  DAU  incorporated  several  other  defense  agency  schools  that  were  active  at 
the  time  to  provide  consistent  training  across  ah  DoD  agencies  as  required  by  DAWIA. 

DAU  certification  is  composed  of  three  components;  education,  training  and 
experience.  Ah  DoD  personnel  filing  acquisition  billets  must  be  certified  in  the  DAU 
career  field  assigned  to  that  position  within  24  months  of  employment.  Training 
requirements  may  be  satisfied  by  completing  DAU  courses,  completing  approved  DAU 
equivalents  such  as  several  NPS  programs,  or  satisfying  the  DAU  fulfillment  program 
which  evaluates  past  experience,  education  and  training  against  competencies  associated 
with  DAU  courses. 

2.  Evolution  of  SPRDE-SE  Career  Field  and  Competencies 

DAU  initially  defined  12  career  paths,  two  of  which  fall  under  the  science  and 
engineering  acquisition  function  and  the  oversight  of  the  Deputy  Assistant  Secretary  of 
Defense  Systems  Engineering  (DASD  [SE]):  Systems  Planning,  Research,  Development 
and  Engineering — Systems  Engineering  (SPRDE-SE);  and  Production,  Quality  and 
Manufacturing  (PQM)  Career  Fields.  Systems  Planning,  Research,  Development  and 
Engineering — Program  Systems  Engineering  (SPRDE-PSE)  was  introduced  in  2007.  The 
impetus  for  the  creation  of  this  additional  career  field  is  discussed  in  more  detail  below. 
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The  DASD  (SE)  serves  as  functional  leader  for  all  three  career  fields.  Supported  by  a 
Functional  Integrated  Product  Team  (FIPT),  he  advises  USD  (AT&L)  with  regard  to 
competencies,  certification  standards  and  position  category  descriptions  (PCDs).  This 
research  will  focus  on  the  certification  requirements  and  curriculum  of  the  SPRDE-SE 
career  field  in  particular. 

Original  competencies  and  course  design  for  the  DAU  career  fields  were  outlined 
by  functional  boards  made  up  of  senior  officials  representing  various  acquisition 
organizations.  These  boards  were  responsible  for: 

•  Ensuring  each  career  field  was  properly  developed  and  implemented 

•  Establishing  the  education,  training,  and  experience  standards  for  career 
paths 

•  Making  recommendations  to  establish  or  disestablish  mandatory  courses 

•  Certifying  annually  to  the  USD  (A)  that  the  curriculum  content  and  quality 
of  each  training  course  were  current  and  complete  (Layton,  2007,  p.  33). 

Using  a  competency-based  model,  DAU  used  Bloom’s  taxonomy  to  provide  the 
framework  for  course  development  (Layton,  2007,  p.  42).  Level  I  certifications  were 
structured  to  reflect  basic  knowledge,  Level  II  built  on  that  basic  knowledge  introducing 
practical  application  and  small  group  scenarios,  and  Level  III  certifications  were  to 
develop  synthesis  and  evaluation  abilities. 

In  2003,  a  joint  summit  of  DoD  and  the  National  Defense  Industrial  Association 
(NDIA)  found  several  critical  areas  of  concern  including  that  “complex  education, 
training  and  recruitment  issues  exist:  alignment  of  career  paths,  role  of  university 
education  and  DAU  training  resources  is  critical”  (Defense  Acquisition  Workforce 
Report  to  Congress,  Appendix  7,  2010,  p.  7).  It  was  discovered  that  while  SE  processes 
were  sound,  application  of  those  processes  was  lacking.  This  report  led  the  USD  (AT&L) 
to  initiate  an  extensive  revitalization  of  DoD  Systems  Engineering.  The  SPRDE  career 
field  curriculum  revision  was  completed  in  2006  with  the  addition  of  four  new  courses: 
SYS  101,  SYS  102,  SYS  203  and  SYS  302,  all  of  which  are  now  required  at  various 
levels  of  certification. 
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3. 


SPRDE-PSE  Career  Field 


In  an  effort  to  further  enhance  systems  engineering  (SE)  certifications,  PCDs  and 
competencies  were  examined  and  another  issue  in  SPRDE  career  development  was 
identified.  For  SPRDE-SE  certification,  DAWIA  requires  a  baccalaureate  or  graduate 
degree  in  a  technical  field  such  as  chemistry,  engineering  or  computer  science.  Billets 
classified  into  the  SPRDE-SE  career  field  were  extremely  diverse  and  included  workers 
from  a  wide  array  of  technical  fields.  Job  descriptions  ranged  from  naval  architect  to 
computer  scientist  to  operations  research  analyst  to  managing  or  supervising  engineers. 
While  this  wide  array  of  talent  contributed  to  the  strength  of  the  workforce  as  a  whole, 
identification  of  Systems  Engineers  possessing  cross  discipline  experience  and  filling 
leadership  roles  at  a  program  systems  integration  level  was  very  difficult. 

To  help  classify  this  specialized  sector  of  the  workforce,  the  SPRDE-PSE  career 
field  was  created  in  2007  (Director,  Human  Capital  Initiatives,  2007,  p.  1).  The  most 
noticeable  difference  from  the  SPRDE-SE  requirements  is  the  experience  component  of 
certification  which  is  twice  that  of  SE  requiring  two,  four,  and  eight  years  for  Levels  I,  II 
and  III  respectively.  Two  additional  DAU  courses  or  equivalents  are  also  required  at  each 
certification  level.  The  addition  of  this  career  field  was  intended  to  help  identify  and 
develop  Systems  Engineers  capable  of  filling  critical  senior  level  positions. 

4.  SPRDE-SE/  PSE  Competency  Model 

From  the  2006  DoD  (AT&L)  Human  Capital  Strategic  Plan  initiative  and  the 
2007  DAU  curriculum  revitalization,  a  SPRDE-SE/PSE  competency  model  emerged. 
Developed  by  the  Center  for  Naval  Analyses  (CNA),  it  was  vetted  by  component  subject 
matter  experts  (SMEs)  in  a  National  Defense  Industrial  Association  (NDIA)  SE 
Education  and  Training  Working  Group.  This  competency  model  was  adopted  by  DAU 
and  published  in  2009  (Development  of  the  SPRDE-SE/PSE  Competency  Model,  2009). 
Consisting  of  29  technical  and  professional  competencies  and  45  competency  elements, 
this  model  was  then  used  as  the  foundation  for  a  competency  model  assessment  that  was 
administered  by  CNA  to  the  SPRDE-SE/PSE  workforce  in  December  2010  through 
March  2011  (Lasley-Hunter,  2011).  This  survey  was  designed  to  gather  data  regarding 
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proficiency,  mission  criticality  and  frequency  of  the  identified  competencies  and  was 
administered  to  both  employees  and  supervisors.  The  methodology  and  results  of  this 
survey  will  be  discussed  further  in  Chapters  II  and  III. 

5.  Naval  Acquisition  Systems  Engineering  Competency  Model 

There  is  consensus  between  NPS  SE  subject  matter  experts  and  Deputy  Assistant 
Secretary  of  the  Navy  (DASN)  Research,  Development,  Test  and  Evaluation  (RDT&E) 
Chief  Engineer  (CHENG)  that  the  required  competencies  and  certifications  for  proficient 
DON  systems  engineers  need  to  be  examined.  As  Chief  Engineer  for  the  Navy,  DASN 
(RDT&E)  desires  a  concrete  definition  of  what  systems  engineers  do  and  how  they 
develop  over  their  careers.  As  part  of  a  DASN  (RDT&E)  Strategic  Initiative,  a  team  from 
NPS’s  Systems  Engineering  Department  is  developing  an  overall  approach  to  the  design 
of  a  naval  systems  engineering  competency  model.  This  model  will  be  referred  to  in  this 
research  as  the  NPS  SE  Competency  Model. 

Any  study  for  the  development  of  a  Naval  Systems  Engineering  Competency 
Model  must  include  SPRDE-SE  learning  objectives,  the  degree  to  which  they  reflect 
actual  competencies  needed  to  perfonn  as  an  acquisition  systems  engineer,  the  impact 
they  are  having  or  could  have  on  SPRDE  acquisition  workforce  members,  and  how 
various  members  fit  into  the  current  DAU  certification  structure.  Even  though  there  is  a 
desire  to  train  both  systems  engineers  and  non-technical  workforce  members  in  some 
aspects  of  systems  engineering  in  the  context  of  systems  acquisition  to  allow  people  in 
technical  fields  that  do  not  fit  into  any  of  the  current  career  paths  access  to  basic 
acquisition  training. 

The  NPS  Team,  led  by  Professor  Clifford  Whitcomb,  first  identified  five  SE 
competency  models: 

•  SE  Workforce  Development  Naval  Underwater  Warfare  Center  (NUWC) 

•  International  Council  on  Systems  Engineering  (INCOSE) 

•  National  Aeronautics  and  Space  Administration  (NASA) 

•  DAU  SPRDE 

•  Boeing  Company 
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Elements  of  all  five  competency  models  were  analyzed,  categorized  and 
duplications  were  eliminated.  The  competency  model  that  emerged  was  mapped  to  the  29 
SPRDE  competencies  plus  two  additional  competencies:  “Systems  Thinking”  and 
“Interpersonal  and  Personal  Characteristics.”  The  NPS  SE  competency  model  team 
identified  2151  KSAs  that  fall  into  these  31  competencies.  The  current  iteration  of  the 
NPS  SE  Competency  Model  only  includes  a  study  to  the  level  of  the  29  competencies 
and  45  elements.  Further  study  is  needed  at  the  level  of  the  DAU  SPRDE  learning 
objectives  to  define  and  integrate  these  into  the  model  as  new  KSAs.  The  evolution  and 
development  of  this  NPS  SE  Competency  Model  will  be  addressed  in  more  detail  in 
Chapter  II. 


6.  Summary 

This  section  has  presented  the  background  of  the  SPRDE-SE  career  field  and 
Competency  Model.  Figure  1  summarizes  the  evolution  of  the  SPRDE-SE  Competency 
Model. 


The  importance  of  training  a  well-qualified,  professional  acquisition  workforce 


has  been  recognized  for  decades.  Certification  processes  have  been  developed  and 

7 


revitalized;  however,  there  is  still  upper  level  concern  that  current  knowledge,  skills  and 
abilities  for  Systems  Engineers  are  not  sufficiently  and  consistently  well-defined  to 
produce  and  support  the  technically  competent  SE  workforce  required  to  support 
acquisition  program  success. 

Current  research  has  identified  a  Naval  Systems  Engineering  Competency  Model 
and  the  KSAs  required  to  perform  successfully  as  a  naval  systems  engineer.  As  a  logical 
progression,  this  project  will  compare  the  NPS  defined  KSAs  and  the  current  DAU 
training  course  objectives  to  determine  the  similarities,  differences,  gaps  and  overlaps 
between  current  training  and  the  capabilities  needed  to  perform  as  a  successful  naval 
systems  engineer. 

B.  PROBLEM  STATEMENT 

Professor  Robert  L.  Hawkins,  Director  of  Curriculum  Development  at  DAU, 
summarized  the  situation: 

The  pure-and-simple  politics  of  acquisition  education  is  that  many  have 
viewed  it  as  a  deficient  system  that  has  failed  to  make  clear  what  all 
students  need  to  learn  and  whether,  in  fact,  they  have  learned  it.... The 
most  fundamental  problem  is  our  schools  are  accountable  only  for 
educational  processes,  not  educational  outcomes.  (Hawkins  and  Granzo, 

1997) 

While  DAU  has  designed  and  implemented  training  programs  that  satisfy 
DAWIA  requirements,  the  outcomes  of  those  training  programs  are  not  necessarily 
targeted  to  the  development  of  highly  educated,  professional,  technically  competent 
systems  engineers  in  the  acquisition  workforce. 

The  overarching  problem  is  whether  or  not  Systems  Engineering  competencies 
are  being  sufficiently  developed  in  the  SE  workforce.  To  address  this  issue,  we  must  first 
define  what  these  competencies  are.  Initially,  this  requires  the  definition  of  what 
competencies  are  needed  to  perform  as  a  proficient  systems  engineer.  This  scope  can 
then  be  narrowed  to  ask  what  competencies  are  needed  to  perform  as  a  proficient  naval 
systems  engineer  and  further  yet  to  what  competencies  are  needed  to  perfonn  as  a 
proficient  naval  acquisition  systems  engineer. 
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Once  these  competencies  are  defined,  they  must  be  compared  to  DAU  SPRDE 
training  currently  in  place  to  identify  gaps  and  overlaps  in  overall  SE  workforce 
development.  As  these  gaps  and  overlaps  in  the  current  training  are  recognized,  progress 
can  be  made  toward  adapting  training  and  thereby  improving  the  quality  of  the  systems 
engineering  workforce. 

C.  RESEARCH  OBJECTIVES 

The  primary  objective  of  this  research  is  to  detennine  the  extent  to  which  the 
DAU  SPRDE-SE  training  objectives  for  DAWIA  Certification  provide  a  basis  for 
defining  KSAs  needed  to  perform  as  a  proficient  acquisition  systems  engineer  in  the 
naval  acquisition  enterprise.  To  support  this  objective,  similarities  and  differences 
between  DAU  SPRDE-SE  training  objectives  and  NPS  SE  Competency  Model  KSAs 
will  be  examined  and  overlaps  and  gaps  will  be  identified.  This  objective  is  illustrated  in 
Figure  2. 

The  secondary  objective  of  this  research  focuses  on  an  internal  analysis  of  the 
DAU  SPRDE  program.  To  this  end,  the  consistency  of  the  DAU  SPRDE  curriculum  will 
be  studied  to  analyze  the  extent  to  which  the  DAU  Enabling  Learning  Objectives  (ELOs) 
support  the  Course  Learning  /  Perfonnance  Objectives  (CL/POs).  Subsequently,  the 
extent  to  which  the  DAU  curriculum  reflects  the  DAU  SPRDE-SE  Competency  Model 
will  be  examined.  The  secondary  research  objective  will  provide  a  high  level  review  of 
the  DAU  SPRDE  program  and  its  learning  objective  consistency. 
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Overlap? 

Gap? 

Similarities? 

Differences? 

Figure  2.  Illustration  of  the  Relationships  defining  the  Research  Objectives 

D.  RESEARCH  QUESTIONS 

The  background  of  this  topic  has  presented  several  issues  that  were  considered  in 
the  formulation  of  the  research  questions.  The  following  research  questions  have  been 
developed  in  response  to  the  previously  outlined  research  objectives.  These  research 
questions  will  guide  the  path  of  this  research  which  will  culminate  in  their  answers  and 
conclusions  provided  in  Chapters  IV  and  V. 

1.  Primary  Research  Questions 

The  primary  research  questions  both  involve  a  comparison  of  the  DAU  SPRDE- 
SE  curriculum  and  the  NPS  SE  Competency  Model. 

Primary  Research  Question  1:  To  what  degree  do  the  DAU  SPRDE-SE  training 
requirements  for  DAWIA  Certification  reflect  the  competencies  needed  to  perfonn  as  a 
proficient  systems  engineer  in  the  naval  acquisition  enterprise  as  identified  by  the  NPS 
SE  Competency  Model? 

Primary  Research  Question  2:  What  are  the  similarities  and  differences  between 
DAU  SPRDE-SE  curriculum  and  NPS  SE  Competency  Model  KSAs?  What  is  the 
overlap  or  gap  between  these  models? 
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2.  Secondary  Research  Questions 

In  studying  the  DAU  curriculum  and  competency  model,  a  set  of  related, 
secondary  research  questions  have  been  developed  that  provide  insight  into  the 
characteristics  of  the  DAU  training  curriculum  and  the  DAU  SPRDE  Competency 
Model.  This  internal  DAU  SPRDE  evaluation  presents  the  secondary  research  questions. 

Secondary  Research  Question  1:  To  what  extent  are  the  DAU  SPRDE-SE  Course 
Learning/Performance  Objectives  supported  by  their  corresponding  Enabling  Learning 
Objectives? 

Secondary  Research  Question  2:  To  what  extent  do  the  DAU  SPRDE-SE  Course 
Learning/  Performance  Objectives  reflect  the  DAU  Competency  Model? 

E.  PURPOSE/BENEFIT 

This  section  defines  Systems  Engineering  and  discusses  the  impact  of  its 
workforce  on  the  acquisition  enterprise.  Several  stakeholders’  initiatives  or  interest  in  the 
development  of  acquisition  workforce  development  will  also  be  examined. 

The  concept  of  Systems  Engineering  was  introduced  in  the  mid-1950s  by  the 
dramatic  upswing  in  technological  complexity.  In  his  book  “Rescuing  Prometheus”, 
Thomas  P.  Hughes  describes  four  post-war  technological  projects  that  have  helped  to 
shape  modern  day  management  styles.  The  Atlas  Project  developed  the  first 
intercontinental  ballistic  missile  and  involved  over  70,000  workers  in  22  different 
industries  along  with  500  military  officers.  Hughes  asserts,  “From  Atlas,  a  new  mode  of 
management  known  as  systems  engineering  emerged  and  has  spread... even  into 
government  agencies”  (Hughes,  1998,  p.  4).  The  SAGE  air  defense  project  was  the  first 
computer  application  that  reached  beyond  computation  to  the  running  of  processes  and 
systems.  SAGE  exemplified  the  Systems  management  structure  that  would  ensure 
integration  and  compatibility  throughout  the  entire  product  life-cycle  from  research  and 
development  through  design  and  performance. 

To  narrow  our  perspective  to  DoD  application,  the  Defense  Acquisition 
Guidebook  (DAG)  defines  Systems  Engineering  as: 
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•  A  methodical  and  disciplined  approach  for  the  specification,  design, 
development,  realization,  technical  management,  operations,  and 
retirement  of  a  system. 

•  SE  applies  critical  thinking  to  the  acquisition  of  a  capability.  It  is  a 
holistic,  integrative  discipline,  whereby  the  contributions  across 
engineering  disciplines... are  evaluated  and  balanced  to  produce  a  coherent 
capability — the  system. 

•  SE  balances  the  conflicting  design  constraints  of  cost,  schedule,  and 
performance  while  maintaining  an  acceptable  level  of  risk  (Defense 
Acquisition  Guidebook,  2013,  p.  4). 

The  DAG  points  out  that  not  only  is  SE  applied  throughout  the  entire  lifecycle  of 
an  acquisition  program,  but  it  also  involves  many  of  the  different  component  processes 
from  design  to  risk  to  budget.  SE  is  integral  to  a  number  of  acquisition  policies  and  it  is 
the  “glue”  that  holds  acquisition  programs  together.  A  professional,  well-trained  SE 
workforce  is  crucial  to  the  success  of  acquisition  programs. 

In  addition  to  the  scope  of  Systems  Engineering,  the  sheer  magnitude  of  the 
SPRDE-SE  workforce  impacts  the  success  of  the  defense  acquisition  process.  Figure  3 
illustrates  the  size  of  the  SPRDE-SE  workforce  in  relation  to  the  other  15  career  fields 
and  to  the  Defense  Acquisition  Workforce  as  a  whole.  SPRDE-SE  represents,  by  far,  the 
largest  component  of  the  AT&L  workforce  with  over  39,000  workers.  The  size  of  this 
sector  magnifies  the  importance  of  adequate  and  accurate  training. 
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Figure  3.  Defense  Acquisition  Workforce  by  Career  Field  (Defense  Acquisition 

Workforce  Update,  2012) 


Another  issue  that  highlights  the  importance  of  this  research  is  the  impending 
departure  of  Baby  Boomers  from  the  AWF.  The  1990s  saw  a  dramatic  drawdown  in  the 
size  of  the  acquisition  workforce.  This  downsizing  included  a  curtailment  of  new  hires, 
which  in  turn  has  left  a  gap  in  the  continuum  of  workforce  age.  Currently,  a  large 
percentage  of  the  AWF  is  close  to  or  already  eligible  for  retirement  and  behind  them  is  a 
vacuum  of  talent  reflecting  a  decade  when  younger  workers  were  not  being  hired.  The 
SPRDE  Workforce  Assessment  found  that  at  least  1 1  percent  of  the  SPRDE  workforce 
will  retire  by  2016  (Lasley-Hunter,  2011,  p.  5).  This  effect  is  even  more  pronounced  at 
upper  levels  of  management  and  presents  yet  another  challenge  to  the  establishment  and 
maintenance  of  a  quality  workforce  (Gansler,  2002).  Effective  certification  and  training 
requirements  identified  by  this  research  are  needed  to  help  fill  this  gap  in  expertise. 
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While  the  scope  of  Systems  Engineering  and  the  size  of  the  SE  workforce  shed  a 
crucial  light  on  the  importance  of  sound  SE  education  and  training,  it  is  also  important  to 
take  note  of  stakeholders  that  are  proponents  of  this  research  area. 

Stephen  Welby,  Deputy  Assistant  Secretary  of  Defense  (DASD)  for  SE,  as  the 
Functional  Leader  for  SPRDE-SE  and  PSE  career  fields,  states: 

One  of  our  greatest  challenges  may  be  in  our  approaches  to  building  great 
people  and  teams  and  improving  how  we  recruit,  grow,  and  mature  the 
technical  and  systems  engineering  professionals  who  will  successfully 
deliver  today  and  tomorrow’s  critical  defense  systems  (Office  of  the 
Deputy  Assistant  Secretary  of  Defense  Systems  Engineering  (ODASD 
[SE],  2013). 

Two  of  the  top  priorities  of  the  DASD  (SE)  are  to  (1)  champion  SE  as  a  tool  to 
improve  acquisition  quality  and  (2)  develop  future  technical  leaders  across  the  acquisition 
enterprise.  One  of  his  strategies  to  accomplish  these  goals  is  to  identify  “workforce 
competencies  crucial  for  executing  systems  engineering  and  production,  quality,  and 
manufacturing  functions  within  acquisition  programs.”  Clearly,  examination  of  these 
competencies  is  paramount  to  achieving  these  goals  (ODASD  [SE],  2013). 

In  2010,  USD  (AT&L),  then  Ashton  Carter,  introduced  the  Better  Buying  Power 
Initiative  in  which  he  highlighted  the  need  for  the  DoD  to  develop  efficiencies  in 
procurement  spending  or  “to  do  more  without  more.”  It  has  been  revised  as  recently  as 
April  2013  and  highlights  seven  focus  areas  including  “Improve  Professionalism  in 
Acquisition  Workforce”  (Defense  Acquisition  University  [DAU],  2012).  Ashton  Carter 
coined  the  phrase,  “Workforce  size  is  important,  but  quality  is  paramount”  (Carter,  2010). 
Carter’s  focus  on  the  importance  of  AWF  development  reflects  on  the  importance  of  this 
research. 

The  DASN  (RDT&E),  a  senior  advisor  to  the  ASN  (RD&A),  oversees  the  DON 
Chief  Engineer  (CHENG)  position  and  is  the  functional  Acquisition  Workforce 
Competency  Leader  for  Engineering.  The  DASN  (RDT&E)  is  actively  involved  in 
engineering  workforce  development  and,  as  a  primary  stakeholder,  is  sponsoring  current 
research  at  NPS.  As  part  of  the  DASN  (RDT&E)  Strategic  Initiative  entitled,  “Initial 
Development  of  Systems  Engineering  Competency  Model,”  the  NPS  research  is  tasked  to 
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determine  precisely  what  it  is  that  systems  engineers  do;  how  that  compares  with  training 
currently  being  provided  through  DAU  certification;  what  the  overlaps  and  gaps  are  and 
how  those  training  gaps  can  be  addressed.  This  research  is  a  vital  part  of  answering  those 
questions.  The  competency  model  being  developed  by  the  NPS  team  is  integral  to  this 
research  and  will  be  discussed  at  length  throughout  this  paper. 

The  ATL  Human  Capital  Strategic  Plan,  initiated  by  USD  (AT&L)  in  2006  and 
revised  in  2010,  identified  “Mission-responsive  workforce  strategies  and  development” 
as  one  of  the  five  goals  (AT&L  HCSP,  2010).  This  research  applies  to  workforce 
development. 

As  the  government  experiences  more  budgetary  constraints,  increasingly  complex 
technological  developments,  reductions  in  the  experience  level  of  the  acquisition 
workforce  due  to  retirement,  getting  Acquisition  right  will  become  more  and  more 
crucial.  The  first  step  to  improving  the  outcomes  is  to  ensure  a  qualified,  professional, 
well-trained  workforce. 

While  DAWIA  and  DAU  have  provided  huge  improvements  in  acquisition 
training,  much  remains  to  be  done.  This  study  will  examine  the  similarities  and 
differences,  gaps  and  overlaps  between  the  knowledge,  skills  and  abilities  that  are 
required  for  SPRDE-SE  certification  and  those  which  have  been  identified  as  being 
necessary  to  perform  successfully  as  a  naval  systems  engineer.  In  providing  insight  into 
areas  of  training  needed  for  Systems  Engineers  beyond  the  DAU  SPRDE-SE  certification 
this  research  will  help  move  towards  the  overlying  goal  of  establishing  a  professional, 
well-trained  acquisition  workforce. 

F.  SCOPE/METHODOLGY 

This  research  will  conduct  a  comparative  analysis  of  the  course  performance  and 
learning  objectives  required  for  DAU’s  SPRDE-SE  Level  I,  II  and  III  Certification  and 
the  knowledge,  skills  and  abilities  defined  by  the  NPS  SE  Competency  Modeling  Team 
as  necessary  to  the  successful  perfonnance  of  a  naval  acquisition  systems  engineer.  First, 
the  DAU  CL/POs  and  ELOs  and  the  NPS  Team’s  KSAs  will  be  categorized  according  to 
Bloom’s  Taxonomy.  A  detennination  will  be  made  as  to  the  extent  of  similarities  and 
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differences  between  the  two  sets  of  data  with  respect  to  Bloom’s  Cognitive  and  Affective 
Domains.  Secondly,  the  DAU  CL/POs  and  ELOs  will  be  compared  directly  with  the  NPS 
KSAs  to  detennine  the  overlap  and/or  gap  between  the  two. 

To  address  the  secondary  research  questions,  the  cognitive  level  categorizations 
of  the  DAU  CL/POs  will  be  compared  directly  to  the  corresponding  ELOs  to  detennine 
to  what  degree  the  ELOs  support  the  cognitive  levels  of  the  CL/POs.  Finally,  the  DAU 
CL/POs  will  be  categorized  according  to  the  DAU  SPRDE  competency  model  and  that 
data  will  be  analyzed  to  detennination  how  well  the  29  DAU  competencies  are 
represented  in  the  DAU  SPRDE-SE  curriculum. 

The  data  used  for  this  research  is  limited  to  the  required  courses  for  SPRDE-SE 
Levels  I,  II  and  III  certification.  The  SPRDE-PSE  certification  requirements  are  inclusive 
of  and  build  upon  the  SPRDE-SE  objectives  by  adding  two  additional  elective  training 
classes  at  each  certification  level  and  doubling  the  required  years  of  experience.  Hence, 
by  analyzing  the  SPRDE-SE  curriculum,  the  foundation  for  both  certifications  is  being 
examined. 

While  DAWIA  certification  requirements  include  elements  of  education,  training 
and  experience,  this  research  will  focus  on  training  component  of  certification.  Further 
research  is  required  to  determine  the  extent  to  which  experience  and  education  satisfies 
the  desired  competencies  for  SE. 

G.  THESIS  STATEMENT 

This  study  will  analyze  and  detennine  to  what  extent  the  DAU  SPRDE-SE 
Certification  curriculum  provides  the  basis  for  defining  KSAs  to  support  the  development 
of  for  Naval  Systems  Engineers.  In  addition,  Bloom’s  Cognitive  Domain  levels  of  the 
objectives  in  the  DAU  curriculum  will  be  compared  internally,  to  examine  the  degree  to 
which  the  DAU  Course  Objectives  are  supported  by  the  curriculum,  as  well  as  to  the  NPS 
SE  Competency  Model  Team’s  KSAs,  to  determine  the  overlap  and/or  gap  in  KSA 
coverage.  Finally,  the  DAU  curriculum  will  be  compared  with  the  DAU  competency 
model  to  determine  the  extent  to  which  all  29  competencies  are  being  addressed. 
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H.  REPORT  ORGANIZATION 


Chapter  I  of  this  research  provides  background  to  the  problem  and  defines  the 
research  objectives  and  questions.  Chapter  II  will  take  a  closer  look  at  the  tools  used  in 
the  analysis  of  this  study.  Chapter  III  will  present  the  data  and  methodology  used  in  the 
analysis.  Chapter  IV  will  discuss  the  findings  of  the  analysis  and  finally,  Chapter  V  will 
reveal  conclusions  and  provide  suggestions  for  further  research. 

I.  SUMMARY 

This  chapter  has  reviewed  the  evolution  of  the  SPRDE-SE  career  field  and  the 
DAU  SPRDE  Competency  Model  as  well  as  the  background  of  the  NPS  SE  Competency 
Model.  A  problem  statement  has  been  refined  into  research  objectives  and  research 
questions  and  the  importance  of  this  study  and  stakeholders  have  been  examined.  Finally, 
the  scope  and  methodology  were  set  forth  and  the  thesis  statement  defined.  Next,  we  will 
take  a  closer  look  at  the  tools  used  in  this  analysis. 
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II.  LITERATURE  REVIEW 


Chapter  I  provided  a  background  for  this  research  by  explaining  the  series  of 
events  that  culminated  in  the  issues  addressed  by  this  research’s  objectives.  This  chapter 
will  focus  on  current  literature  surrounding  the  tools  used  in  our  analysis.  First,  we  will 
take  a  closer  look  at  competency  models  in  general,  how  they  are  developed  and  how 
they  are  assessed.  Next,  two  competency  models  in  particular  will  be  examined;  the  DAU 
SPRDE  Competency  Model  and  the  NPS  SE  Competency  Model.  Finally,  we  will  review 
Bloom’s  Taxonomy  of  Educational  Objectives  and  why  it  was  chosen  as  a  comparative 
framework  for  this  research. 

A.  COMPETENCY  MODELS  AND  ASSESMENTS 

This  section  will  define  competencies  and  competency  models  and  discuss  their 
characteristics  and  uses.  The  DAU  SPRDE-SE  Competency  Model  will  then  be  examined 
in  more  detail  to  gain  a  deeper  understanding  of  this  element  of  this  study’s  analysis. 

The  Office  of  Personnel  Management  (OPM)  describes  a  competency  as  “an 
observable,  measurable  pattern  of  skills,  knowledge,  abilities,  behaviors  and  other 
characteristics  that  an  individual  needs  to  perform  work  roles  or  occupational  functions 
successfully”  (Lasley-Hunter,  2011,  p.  8).  The  Cambridge  English  Dictionary  provides  a 
more  succinct  definition:  “an  important  skill  that  is  needed  to  do  a  job.” 

The  Department  of  Labor,  Employment  and  Training  Administration  defines  a 
competency  model  as  “a  collection  of  competencies  that  together  define  successful 
performance  in  a  particular  work  setting  (Develop  a  Competency  Model,  2013). 

Competency  models  are  valuable  for  many  reasons.  Competency  models  provide 

the  framework  for  competency  assessments.  Common  uses  include  the  support  of  many 

Human  Resource  functions  such  as  recruitment  and  hiring,  job  training  and  performance 

evaluations  and  career  development.  Competency  assessments  provide  a  framework  to 

demonstrate  professionalism  and  instill  confidence  in  both  workers  and  customers. 

Another  interesting  application  is  the  realistic  reflection  of  individual  competency. 

Studies  have  shown  that  incompetent  people  see  themselves  as  being  more  competent 
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than  they  are  while  competent  people  see  themselves  as  less  competent  than  they  are 
(Kruger,  1999).  Accurate  competency  assessments  allow  management  to  allocate 
personnel  resources  in  a  more  realistic  and  effective  manner.  Competency  assessment  is 
“the  key  to  an  organization’s  overall  capability  improvement”  (Holt,  201 1,  p.  xvi). 

Now  that  we  know  what  competency  models  are  and  what  they  are  used  for,  we 
will  examine  their  characteristics  and  components.  Competency  models  can  be  specific  or 
general  with  applications  ranging  from  a  specific  job  to  a  job  group  to  an  occupation  to 
an  industry.  Most  competency  models  use  some  type  of  representative  visual  diagram  and 
include  a  reference  to  different  levels  of  mastery,  such  as  entry  level,  mid-level  or  highly 
experienced.  They  are  typically  made  up  of  competency  names  with  detailed  definitions 
and  detailed  descriptions  of  activities  or  behaviors  associated  with  each  competency, 
sometimes  referred  to  as  KSAs  or  elements  (U.S.  Dept,  of  Labor,  2013).  A  competency 
element  is  a  behavioral  statement  describing  a  cluster  of  related  tasks  that  results  in  an 
outcome  of  high  value  (SPRDE-SE/PSE  Competency  Assessment  Employee  Users 
Guide,  2010). 

The  OPM  defines  KSAs  as  the  following: 

•  Knowledge,  Skills  and  Abilities  (KSAs)  are  the  attributes  required  to 
perform  a  job  and  are  generally  demonstrated  through  qualifying  service, 
education  or  training. 

•  Knowledge  is  an  organized  body  of  information  applied  directly  to  the 
performance  of  a  function.  \ 

•  Skill  is  an  observable  competence  to  perform  a  learned  psychomotor  act. 

•  Ability  is  the  competence  to  perform  an  observable  behavior  or  a  behavior 
that  results  in  an  observable  product. 

Beyond  these  basic  components,  good  competency  models  also  include  other 
traits.  Typically,  they  combine  skills  from  several  different  sources  and  the  frameworks 
have  evolved  through  many  iterations.  They  apply  across  different  levels  of  proficiency 
but  are  still  simple  and  easy  to  understand.  Finally,  Holt  characterizes  good  competency 
models  as  those  that  focus  on  specific,  technical  skills  rather  than  general,  professional, 
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human  skills  (Holt,  2011).  Others  in  the  field  believe  that  while  soft  skills  may  not  belong 
in  a  core  technical  competency  model,  they  are  needed  and  belong  in  a  competency 
model  of  their  own. 

The  U.S.  Department  of  Labor,  Employment  and  Training  Administration  (ETA) 
User  Guide  to  Competency  Models  (Develop  a  Competency  Model,  2013)  shows  five 
steps  to  developing  a  competency  model  as  shown  in  Figure  4. 


STEP  1:  Gather  background  information 


Existing  frameworks  and  models  are  analyzed,  organized  and  evaluated  to 

determine  affinities 


STEP  2:  Develop  a  draft  competency  model  framework 


Themes  and  patterns  in  the  existing  information  are  identified,  reorganized  and 

a  draft  model  is  developed. 


STEP  3:  Gather  feedback  from  SMEs 


Review  is  requested  to  determine  if  the  framework  reflects  appropriate 
competencies;  if  any  competencies  are  missing  and  if  any  terminology  changes 

need  to  be  made. 


STEP  4:  RcTrAe  the  competency  framework 


Development  of  a  competency  model  is  an  iterative  process.  Revisions, 
additions,  deletions,  and  reorganization  occur  at  this  step. 


STEP  5:  Validate 


Ensure  acceptance  by  a  target  community  of  users. 


Figure  4.  Competency  Model  Development  Steps 


Note  that  steps  3  and  4  are  repeated  until  the  SMEs  and  the  development  team 
agree  that  the  model  is  an  all-inclusive  representation  of  required  KSAs.  Later  in  this 
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section,  two  competency  models  will  be  examined;  the  DAU  SPRDE  Competency  Model 
and  the  NPS  SE  Competency  Model,  and  we  will  see  how  these  five  developmental  steps 
have  been  implemented. 

Competency  models  provide  the  framework  for  competency  assessments.  A 
competency  assessment  is  an  application  or  comparison  of  a  competency  model  to  a 
worker’s  or  workforce’s  capabilities.  Good  competency  assessments  are  repeatable, 
providing  for  accurate  comparison;  measurable;  based  on  best  practices;  traceable  to 
standards  and  frameworks;  transferable  between  frameworks  and  tailorable.  Ideally, 
competency  assessments  will  be  as  flexible  and  scalable  as  possible.  Competency 
assessment  output  is  a  worker  profile.  This  profile  is  not  a  “pass/fail”  audit  but  rather  it 
paints  an  overall  picture  of  an  employee’s  strengths  and  weaknesses,  gaps  in  training  and 
the  level  of  their  career  development  (Holt,  201 1,  p.  8). 

This  section  has  provided  an  overview  of  competency  models  and  competency 
assessments;  what  they  are  used  for,  what  they  consist  of,  how  they  are  developed  and 
what  defines  a  good  one.  Now  we  will  examine  the  two  competency  models  used  in  the 
analysis  of  this  research  to  better  understand  these  particular  models  and  their  impact  on 
this  research. 

B.  DAU  SPRDE  COMPETENCY  MODEL  DEVELOPMENT  AND 

ASSESSMENT 

Because  the  DAU  SPRDE  Competency  Model  plays  a  significant  role  in  this 
research,  it  warrants  a  deeper  examination.  In  the  analysis  for  the  fourth  research 
question,  a  direct  comparison  is  made  to  see  if  the  DAU  SPRDE-SE  curriculum  is 
representative  of  and  supports  the  DAU  SPRDE  Competency  Model.  The  DAU  SPRDE 
Competency  Model  is  also  the  foundation  or  original  framework  for  the  NPS  SE 
Competency  Model  as  will  be  discussed  in  the  next  section.  The  primary  research 
objective  involves  comparisons  with  the  NPS  SE  Competency  Model  and  therefore  that 
analysis  will  be  indirectly  affected  by  the  DAU  SPRDE  Competency  model  as  well. 
Because  the  DAU  SPRDE  Competency  Model  is  woven  throughout  this  research,  it  is 
important  to  understand  its  development  and  characteristics  before  we  begin  to  use  it  as  a 
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tool.  This  section  will  also  examine  the  findings  of  the  associated  competency  assessment 
and  the  impact  these  characteristics  and  findings  have  on  our  research. 

As  mentioned  in  Chapter  I,  the  DAU  SPRDE  Competency  model  was  a  result  of 
SE  curriculum  revitalization  efforts  and  the  2006  DoD  (AT&L)  Human  Capital  Strategic 
Plan  initiative.  The  formulation  of  the  DAU  SPRDE  Competency  Model  is  illustrative  of 
the  developmental  steps  presented  previously  in  Figure  4.  Initially  a  small  group  of  career 
field  experts  and  leaders  identified  behaviors,  knowledge,  skills  and  characteristics 
required  to  be  a  successful  SPRDE  employee  and  created  a  framework.  The  group  was 
then  expanded  to  include  successful  workforce  members  and  the  model  was  analyzed, 
added  to,  deleted  from  and  compared  directly  to  job  responsibilities.  Further  refinements 
and  revisions  finally  resulted  in  the  competency  model  shown  in  Figure  5  that  includes  29 
competencies  categorized  into  3  competency  units;  analytical,  technical  management  and 
professional.  The  model  also  includes  45  unique  elements  which  apply  to  the  various 
competencies.  A  full  version  of  this  model,  including  elements,  is  available  in  Appendix 
A  (Lasley-Hunter,  2011,  pp.  12,  83-87). 


Competencies  in  the  Analytical 
Unit  of  Competence 

Competencies  in  the  Technical 
Management  Unit  of 
Competence 

Competencies  in  the 
Professional  Unit  of 

Competence 

l 

Technical  Basis  for  Cost 

14 

Decision  Analysis 

26 

Communications 

2 

Modeling  and  Simulation 

15 

Technical  Planning 

27 

Problem  Solving 

3 

Safety  Assurance 

16 

Technical  Assessment 

28 

Strateqic  Thinkinq 

4 

Stakeholder  Requirements 
Definition  (Requirements 
Development) 

17 

Configuration  Management 

29 

Professional  Ethics 

5 

Requirements  Analysis  (Logical 
Analysis) 

18 

Requirements  Management 

6 

Architectural  Design  (Design 
Solution) 

19 

Risk  Management 

7 

Implementation 

20 

Technical  Data  Management 

8 

Integration 

21 

Interface  Management 

9 

Verification 

22 

Software  Engineering 

10 

Validation 

23 

Acquisition 

11 

Transition 

24 

System  Engineering  Leadership 

12 

System  Assurance 

25 

System  of  Systems 

13 

Reliability.  Availability,  and 
Maintainabilitv  (RAM) 

Figure  5.  DAU  SPRDE  Competency  Model 


Two  characteristics  of  this  model  are  particularly  worthy  of  note.  First,  this  model 
includes  both  technical  and  professional  competencies.  Technical  competencies  are 
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functional  and  specific  to  a  career  field.  Professional  competencies,  such  as  leadership, 
communication,  problem  solving  and  ethics,  apply  to  all  engineering  career  fields  and  are 
also  referred  to  as  “soft  skills.”  In  the  previous  section,  it  was  noted  that  good 
competency  models  emphasize  technical  skills  rather  than  “soft”  professional  skills.  This 
model  characteristic  will  be  discussed  in  greater  depth  in  the  next  section  as  we  discuss 
the  development  of  the  NPS  SE  Competency  Model.  Suffice  it  now  to  understand  that  4 
of  the  29  competencies  in  the  DAU  SPRDE  Competency  Model  reflect  these  “soft  skills” 
or  professional  competencies.  Secondly,  this  model  does  not  address  different  levels  of 
proficiency,  which  is  another  characteristic  of  good  competency  models.  There  is  no 
distinction  between  the  competencies  needed  at  entry-level  as  opposed  to  the 
competencies  needed  by  an  expert  in  the  SPRDE  field. 

In  December  2010,  at  the  fourth  stage  of  DAU  SPRDE  Competency  Model 
development  and  validation,  the  CNA  administered  a  competency  model  assessment  to 
the  SPRDE  workforce.  It  important  here  to  clarify  tenninology  that  might  be  confusing. 
As  discussed  previously,  competency  models  are  used  for  competency  assessments  which 
measure  worker  or  workforce  capabilities.  This  assessment  was  an  evaluation  of  the 
DAU  SPRDE  Competency  Model  itself,  rather  than  the  workforce. 

The  CNA  survey  was  designed  to  determine  self-assessed  proficiency,  criticality, 
and  the  frequency  of  performance  of  each  of  the  29  competencies  and  45  elements  in  the 
DAU  SPRDE  Competency  Model.  Data  on  proficiency  and  criticality  was  also  collected 
from  supervisors;  however,  a  low  response  rate  resulted  in  the  exclusion  of  this  data  in 
the  final  analysis.  Additionally,  demographic  information  was  collected  from  the 
respondents  to  assist  in  other  assessment  goals,  such  as  the  development  of  a  workforce 
profile  and  workforce  size  and  proficiency  projections.  The  final  analysis  of  the  DAU 
SPRDE  Competency  Assessment  was  reported  by  CNA  in  2011.  Several  of  these 
findings  are  worthy  of  note  in  the  context  of  this  research. 

First,  with  regard  to  validity,  the  assessment  was  completed  by  over  10,000 

employees  representing  approximately  27  percent  of  the  SPRDE  workforce.  Although  it 

was  detennined  that  the  number  of  employee  responses  is  “highly  representative”  of  the 

overall  SPRDE  workforce  population,  the  size  of  the  sample  should  be  taken  into 
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consideration  when  applying  these  results  to  the  entire  SPRDE  workforce  (Lasley- 
Hunter,  2011,  pp.  1-2,  14).  There  seems  to  be  no  definitive  evidence  as  to  what  segment 
of  the  workforce  responded  to  the  assessment;  whether  the  respondents  were  a  true 
random  sample  or  whether  the  respondents  were  possibly  a  pool  of  workers  that  may 
have  been  less  engaged,  less  pressured,  or  had  fewer  responsibilities  in  their  jobs  and 
consequently  had  more  time  to  complete  an  online  assessment.  The  low  rate  of 
supervisory  response  seems  to  support  this  possibility.  Because  no  data  was  recorded  as 
to  the  makeup  of  the  respondents,  the  findings  of  this  assessment  may  have  been 
somewhat  skewed. 

Secondly,  as  part  of  the  demographic  analysis,  it  was  found  that  over  half  of  the 
SPRDE  workforce  has  obtained  a  Master’s  degree  or  higher  and  that  87  percent  of  the 
highest  degrees  obtained  were  in  science,  technology,  engineering,  or  math  (Lasley- 
Hunter,  2011,  p.  30-31).  The  significance  of  this  finding  stems  from  the  fact  that 
education  is  one  of  three  components  of  DAWIA  certification  in  conjunction  with 
training  and  experience.  The  scope  of  this  paper  is  limited  to  examination  of  the  training 
element  and  how  it  relates  to  needed  competencies.  Overarching  research  evaluating 
workforce  capability  would  need  to  take  into  consideration  the  high  education  levels  of 
this  workforce  as  well  as  skills  gained  on  the  job.  The  assessment  also  found  that  57 
percent  of  the  SPRDE  workforce  is  certified  at  Level  III  which  suggests  that  perhaps  this 
“expert”  Level  is  too  easily  attained.  Higher  levels  of  certification  may  be  warranted  and/ 
or  the  content  of  these  Levels  may  need  readjustment. 

Next,  respondents  from  the  SE  workforce  community  rated  each  of  the  29 
competencies  with  regard  to  criticality  and  frequency  of  use.  These  parameters  were 
combined  to  deduce  “importance”.  Only  seven  of  the  29  competencies  were  rated  as 
“highly  important;”  eight  as  of  “medium  importance;”  and  14,  or  almost  half,  as  “low 
importance.”  Competencies  identified  as  highly  important  to  Systems  Engineering 
respondents  are  shown  in  Table  1. 

Note  that  four  of  the  seven  competencies  rated  as  highly  important  are 

professional  or  “soft  skills”  which  apply  across  all  career  fields.  Of  the  25  analytic  and 

technical  management  competencies  specific  to  Systems  Engineering,  only  three  (or  12 
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percent)  were  rated  as  being  highly  important.  Even  though  Lasley-Hunter  states  that  the 
analysis  “suggests  that  the  SPRDE  competency  model  captures  the  competencies  most 
pertinent  to  the  Systems  Engineering,”  (Lasley-Hunter,  2011,  p.  3)  the  fact  that  only  12 
percent  of  the  technical  competencies  were  rated  as  “highly  important”  would  imply 
otherwise.  This  result  illustrates  that  the  competencies  represented  in  the  DAU  SPRDE 
Competency  Model  may  not  accurately  reflect  the  important  competencies  needed  in  the 
SE  workforce. 

Table  1.  DAU  SPRDE  Competencies  Rated  As  Being  “Highly  Important”  (After  Lasley- 

Hunter,  201 1,  p.  36) 


Analytical  Unit 

of  Competency 

Competency  5:  Requirements  Analysis 

Competency  7:  Implementation 

Competency  8:  Integration 

Professional  Unit 

of  Competency 

Competency  24:  Systems  Engineering  Leadership 

Competency  26:  Communication 

Competency  27:  Problem  Solving 

Competency  29:  Professional  Ethics 

Another  interesting  finding  was  the  analysis  of  those  competencies  that  were  rated 
as  being  “N/A”  or  not  applicable.  This  refers  to,  in  most  cases,  competencies  not  needed 
on  the  job  or,  in  fewer  cases,  competencies  that  the  respondents  were  unaware  of  or 
unexposed  to.  Each  of  the  45  elements  received  N/A  ratings  ranging  from  1  to  38  percent 
with  the  elements  for  Acquisition  and  Software  Development  receiving  the  highest 
percentages  of  N/A  ratings  (Lasley-Hunter,  201 1,  p.  42).  This  finding  will  come  into  play 
in  the  analysis  of  our  third  research  objective  when  the  DAU  CL/POs  are  directly 
compared  to  the  DAU  SPRDE  Competency  Model  to  show  which  competencies  the 
DAU  curriculum  addresses. 

Finally,  the  results  suggest  that  SE  respondents  have  a  proficiency  level  of 
intermediate  to  advanced  for  all  competencies  rated  as  highly  important  and  intermediate 
proficiency  in  most  other  competencies  (Lasley-Hunter,  2011,  p.  4).  Recall  the  study 
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referenced  earlier  by  Kruger  and  Dunning  that  revealed  how  highly  competent  people 
tend  to  underestimate  their  level  of  competency  while  incompetent  people  tend  to 
overestimate  their  level  competency.  This  theory  could  have  a  bearing  on  the  proficiency 
findings  because  they  were  based  on  self-assessment  and  therefore,  possibly  skewed. 

In  conclusion,  the  DAU  SPRDE  Competency  Model  was  credibly  developed  and 
vetted.  The  resultant  SPRDE  Competency  Assessment,  completed  in  2011,  provides 
interesting  insight  into  the  SPRDE  workforce;  however,  low  participation  leaves 
questions  as  to  how  well  the  results  reflect  the  entire  workforce.  Both  high  education  and 
high  certification  levels  were  exhibited,  indicating  that  levels  of  certification  might  be 
adjusted  or  added  to.  Competency  23,  Acquisition,  received  one  of  the  highest  “not 
needed  on  job”  ratings  and  this  will  have  an  impact  as  we  develop  our  analysis.  Finally, 
because  the  proficiency  ratings  were  based  on  self-assessment,  they  may  not  accurately 
reflect  the  proficiency  of  the  workforce  as  a  whole. 

Next,  we  will  look  further  into  the  development  and  characteristics  of  another 
competency  model  used  extensively  in  this  research,  the  NPS  SE  Competency  Model. 

C.  NPS  SE  COMPETENCY  MODEL 

The  research  goals  for  the  overall  AT&L  Competency  Program  are: 

•  AT&L  Goal — 1:  Define  the  competencies  required  to  deliver  (needed) 
capabilities. 

•  AT&L  Goal — 2:  Assess  the  workforce  to  identify  current  and  future  gaps 
(Lasley-Hunter,  201 1,  p.  8). 

In  response  to  these  goals,  the  DASN  (RDT&E)  CHENG  was  not  completely 
convinced  by  the  CNA  study  that  the  DAU  SPRDE  Competency  Model  accurately 
reflected  the  skills  needed  by  systems  engineers  in  particular.  Dr.  Clifford  A.  Whitcomb, 
Chairman  of  the  Systems  Engineering  Department  at  NPS,  was  tasked  with  leading  a 
research  team  to  develop  a  comprehensive  approach  to  the  design  of  a  Naval  Systems 
Engineering  Competency  Model.  This  primary  nucleus  of  research  has  spurred  several 
branches  of  associated  student  theses,  including  this  project.  Other  related  theses  are 
noted  in  Chapter  V. 
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The  NPS  Team  found  Holt’s  Pragmatic  Guide  to  Competency  to  be  a  good 
reference  and  roadmap  in  the  formulation  of  their  model.  In  fact,  one  of  the  five  models 
used  in  the  NPS  research,  INCOSE,  was  evaluated  in  this  guide.  Holt  is  a  proponent  of 
combining  appropriate  pieces  of  existing  frameworks  into  a  new,  unique  framework  that 
is  applicable  to  a  new  field  (Holt,  2011,  p.  7-8).  This  is  precisely  what  the  NPS  Team 
did.  Initial  development  of  the  NPS  SE  Competency  Model  began  with  the  identification 
of  five  previously  existing  SE  competency  models: 

•  Naval  Undersea  Warfare  Center  (NUWC),  Newport  SE  Workforce 
Development  Model 

•  International  Council  on  Systems  Engineering.  United  Kingdom  Chapter 
(INCOSE  UK)  SE  Competency  Model 

•  National  Aeronautics  and  Space  Administration  (NASA)  SE  Competency 
Model 

•  DAU  SPRDE  SE  Competency  Model 

•  Boeing  SE  Competency  Model 

Elements  of  these  five  models  were  categorized  based  on  affinity,  mapped  to  the 
29  DAU  competencies  and  duplicates  were  eliminated.  The  resultant  model  contained 
2,151  KSAs  in  31  competencies.  29  of  these  competencies  were  identical  to  the  DAU 
SPRDE  Model.  The  two  additional  competencies  identified  were  “30.0  Systems 
Thinking”  and  “31.0  Interpersonal  and  Personal  Characteristics.” 

As  discussed  previously  in  this  research,  a  good  competency  model  focuses  on 
technical  rather  that  “soft  skills.”  The  second  version  of  the  NPS  SE  Competency  Model 
removed  the  professional  skills  and  related  KSAs  from  the  framework.  The  intent  was 
that  these  professional  competencies  would  be  applied  to  a  separate,  more  generally 
applied  competency  model  rather  than  one  specifically  for  systems  engineers.  This  update 
effectively  removed  499  KSAs  and  six  associated  competencies.  As  a  result,  the  second 
version  of  the  NPS  SE  Competency  Model  contained  25  technical  or  core  competencies 
and  1652  associated  KSAs. 

Competencies  removed  in  Version  2  of  the  NPS  SE  Competency  Model: 

•  24.0  Systems  Engineering  Leadership 

•  26.0  Communications 
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•  27.0  Problem  Solving 

•  28.0  Strategic  Thinking 

•  29.0  Professional  Ethics 

•  31.0  Individual  and  Interpersonal  Characteristics 

To  address  the  application  of  these  competencies  across  proficiency  levels,  each 
KSA  was  mapped  to  one  of  three  specific  career  levels  designated  as  SE-1  Entry  Level, 
SE-2  Journey  Level  or  SE-3  Expert  Level.  This  was  accomplished  by  first  rating  each 
KSA  according  to  Bloom’s  Taxonomy  of  Educational  Objectives  which  is  discussed  in 
more  detail  later  in  this  section.  Specifically,  the  NPS  team  used  the  version  of  Bloom’s 
Taxonomy  illustrated  in  the  Graduate  Reference  Curriculum  for  Systems  Engineering 
(GRCSE),  which  is  shown  in  Table  2.  While  several  interpretations  and  one  revision  of 
Bloom’s  Taxonomy  exist,  this  GRCSE  version  was  chosen  because  it  was  the  most 
closely  associated  with  Systems  Engineering  and  the  list  of  verbs  was  the  most  succinct 
and  had  the  least  repetition.  This  further  strengthened  the  NPS  Model  with  the  inclusion 
of  yet  another  valid  source. 

Bloom’s  Taxonomy  provides  hierarchical  outcome  categories  or  levels  that  range 
from  simple  to  complex  thought  processes.  Once  the  Bloom’s  level  for  each  KSA  was 
identified,  the  KSAs  in  each  competency  were  divided  into  the  three  career  levels  by 
assigning  the  KSAs  with  lower  Bloom’s  level  ratings  to  the  SE-1  Entry  Level,  the  KSAs 
with  intermediate  Bloom’s  level  ratings  to  SE-2  Journey  Level  and  the  KSAs  with  higher 
Bloom’s  level  ratings  to  the  SE-2  Expert  Level. 

Finally,  the  competencies  were  also  categorized  as  to  whether  they  would  be 
developed  by  “Education  and  Training;”  “On  the  Job  Experience;”  or  “Professional 
Development.”  An  example  of  this  model  showing  Competency  1.0  Technical  Basis  for 
Cost  is  illustrated  in  Appendix  B. 

The  third  and  current  iteration  of  the  NPS  SE  Competency  Model  was  actually 
precipitated  by  the  research  addressed  in  this  paper.  As  an  initial  part  of  the  analysis  for 
this  project,  654  Course  Learning/  Perfonnance  Objectives  and  Enabling  Learning 
Objects  for  seven  DAU  SPRDE-SE  Level  III  required  courses  were  identified.  These 
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CL/POs  and  ELOs  were  then  defined  as  KSAs  and  added  to  the  NPS  Model  as  shown  in 
Figure  6.  They  were  also  mapped  to  competencies  in  the  NPS  Model  and  to  the  GRCSE 
Bloom’s  levels. 


NASA 

SC  Competency 
Model 
DAU  SPRDE 
SC  Competency 
Model 


NUWC  Newport 
SC  Workforce 
Development 


INCOSCUK 
SC  Competency 

Model  g-  .. 

Boeing 
am  pet  en 
Model 


Adds  3  DAU  Competencies 
Rewords  CL/POs  and  ELOs  as 
KSAs 


Removes  Duplicates 
Re  moves  47  DAU  KSAs 
Removes  1  NPS  KSA 


Figure  6.  Evolution  of  NPS  SE  Competency  Model 


Incorporation  of  the  DAU  CL/POs  and  ELOs  into  the  NPS  SE  Competency 
Model  required  considerable  manipulation  before  the  final  third  version  emerged.  First, 
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the  new  KSAs  were  reworded  to  incorporate  the  same  Bloom’s/  GRCSE  verbs  that  are 
used  in  the  rest  of  the  NPS  Model.  Six  duplicates  were  discovered  and  discarded.  Next, 
KSAs  associated  with  the  professional  competencies  (Competencies  24.0,  26.0,  27.0, 
28.0,  29.0  and  31.0)  partitioned  into  a  separate  model  from  the  SE  model.  The  Team  then 
agreed  that  35.  0  (Domain  Specialization)  should  be  discarded  as  it  didn’t  apply  explicitly 
to  systems  engineering.  That  removed  another  10  DAU  KSAs  as  well  as  one  KSA  from 
version  2  of  the  NPS  SE  Competency  Model.  A  total  of  607  DAU  KSAs  and  three 
competencies  (32.0  Financial  Management,  33.0  Contracting  and  34.0  Facilities 
Engineering)  were  ultimately  added  to  the  NPS  SE  Competency  Model. 

The  resultant  and  current  version  of  the  NPS  SE  Competency  Model  includes 
2,257  KSAs  in  28  competencies.  Figure  6  show  a  graphic  illustration  of  the  evolution  of 
the  NPS  SE  Competency  Model,  version  three  (v.3)  is  used  in  the  analysis  of  this 
research. 

In  conclusion,  the  NPS  SE  Competency  Model  has  many  traits  of  a  good 
competency  model.  It  is  based  on  several  sources  and  has  evolved  through  many 
iterations.  It  applies  across  different  proficiency  levels  and  is  easy  to  understand.  Finally, 
the  NPS  SE  Competency  Model  is  based  on  technical  skills  rather  than  “soft”  or 
professional  skills.  For  these  reasons,  the  NPS  SE  Competency  Model  is  valid  tool  for  the 
comparative  analysis  performed  in  this  research. 

D.  BLOOM’S  TAXONOMY  OF  EDUCATIONAL  OBJECTIVES 

This  section  will  examine  yet  another  tool  used  in  the  analysis  of  this  research: 
Bloom’s  Taxonomy.  The  research  objectives  for  this  project  involve  a  comparative 
analysis  between  the  DAU  SPRDE-SE  curriculum  and  the  NPS  SE  Competency  Model 
as  well  as  DAU  SPRDE-SE  curriculum  and  the  SPRDE  Competency  Model.  To 
implement  these  comparisons,  a  framework  was  needed  to  ensure  that  the  research  was 
comparing  “apples  with  apples.”  DAU  initially  used  Bloom’s  Taxonomy  to  provide  the 
framework  for  course  development.  The  NPS  SE  Competency  Model  also  had  used 
Bloom’s  Taxonomy  to  classify  KSAs  into  career  levels.  Therefore,  the  popular  and  well 
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used  Bloom’s  Taxonomy  was  a  logical  choice.  This  section  will  look  closer  at  the 
background  of  the  Taxonomy,  what  it  consists  of,  and  why  it  is  a  good  comparative 
framework. 

Bloom’s  Taxonomy  of  Educational  Objectives  is  a  classification  of  learning 
objectives  originally  developed  to  support  communication  between  examiners.  Although 
the  Taxonomy  takes  Bloom’s  name,  it  was  a  developed  by  a  consortium  of  34  educators 
who  contributed  by  attending  a  series  of  conferences  between  1949  and  1953.  As  one  of 
the  first  classification  frameworks  for  learning  processes  and  educational  goals,  it  is  one 
of  the  most  widely  applied  and  most  often  cited  references  in  education.  “When  an 
instructor  desires  to  move  a  group  of  students  through  a  learning  process  utilizing  an 
organized  framework,  Bloom’s  Taxonomy  can  prove  helpful”  (Forehand,  2005). 

While  Bloom’s  Taxonomy  has  had  a  wide  range  of  application,  one  of  the  most 
common  uses  has  been  “to  classify  curricular  objectives  and  test  items  in  order  to  show 
the  breadth,  or  lack  of  breadth,  of  the  objectives  and  items  across  the  spectrum  of 
categories”  (Krathwohl,  2002).  This  applies  precisely  to  our  research  objectives. 

Bloom’s  Taxonomy  is  a  multi-dimensional  framework  consisting  of  three 
domains:  Cognitive;  Affective  and  Psychomotor.  Loosely,  these  can  be  thought  of  as 
knowing/head;  feeling/heart;  and  doing/hands.  Each  of  these  domains  is  comprised  of 
several  hierarchical  levels.  Each  level  requires  prior  assimilation  of  the  knowledge  and 
skills  at  next  lower  level.  The  idea  of  the  Taxonomy  was  to  motivate  educators  to  use  all 
three  Domains  to  create  a  more  comprehensive  educational  approach. 

The  Cognitive  Domain  was  the  first  Domain  developed,  is  the  most  widely  used 
and  best  understood.  Cognition  is  defined  as  a  group  of  mental  processes  that  range  from 
remembering  to  learning,  reasoning,  problem  solving  and  decision  making.  This  domain 
is  knowledge  based  and  is  highlighted  in  traditional  education.  It  includes  six  levels: 
Knowledge,  Comprehension,  Application,  Analysis,  Synthesis  and  Evaluation  as  shown 
in  Table  2. 

•  Knowledge  is  the  lowest  of  the  cognitive  levels  and  involves 
remembering,  the  recall  of  information.  The  analysis  in  Chapter  III  will 
support  the  fact  that  this  cognitive  level  is  common  in  education.  Results 
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are  typically  right  or  wrong  and  opinion  and  judgment  don’t  come  into 
play.  This  type  of  outcome  is  straightforward,  making  it  simpler  to  test  and 
define. 

•  Comprehension  is  the  understanding  or  interpretation  of  information. 

•  Application  is  using  information  to  solve  a  problem;  applying  knowledge 
to  a  task. 

•  Analysis  is  the  breaking  down  of  information  into  parts  and  understanding 
how  parts  are  related  and  organized. 

•  Synthesis  is  the  combining  of  parts  to  create  a  new  whole. 

•  Evaluation  is  judging  the  value  or  worth  of  information  and  ideas 

Analysis  of  an  educational  curriculum  typically  shows  a  heavy  emphasis  on  the 

cognitive  domain  and  specifically,  learning  objectives  that  involve  recall  and 
memorization  and  that  fall  into  the  lower  Bloom’s  categories.  This  is  in  direct  conflict 
with  most  educational  goals  that  favor  the  higher  cognitive  levels  of  understanding, 
applying,  analyzing  and  creating.  Systems  engineering,  in  particular,  is  involved  with 
analyzing  and  creating.  Educators  also  tend  to  give  even  less  attention  to  the  affective 
domain  which  we  will  examine  next. 

The  affective  domain  has  to  do  with  emotions,  feelings  and  attitudes.  Objectives 
describe  growth  in  awareness,  attitude,  emotion,  changes  in  interest,  judgment  and  the 
development  of  appreciation.  While  most  educators  do  not  include  the  affective  domain 
in  their  curriculum,  and  in  fact  this  domain  is  completely  absent  from  the  DAU  SPRDE 
curriculum,  these  outcomes  are  especially  critical  to  the  success  of  systems  engineers. 
Systems  engineers  lead  systems  projects,  negotiate  outcomes  with  a  diverse  group  of 
stakeholders,  make  value  judgments  and  must  have  the  ability  to  deliberately  take  the 
systems  perspective  (GRCSE,  2011,  p.  95).  These  are  all  affective  outcomes.  Almost  29 
percent  of  the  NPS  SE  Competency  Model’s  KSAs  fall  into  this  affective  domain.  The 
five  levels  of  the  affective  domain  are  shown  in  Table  3. 

Because  Bloom’s  Taxonomy  has  been  around  for  over  50  years,  there  have  been 
many  interpretations.  Indeed  a  simple  search  of  the  Internet  will  yield  dozens  of  slightly 
different  versions  which  have  been  condensed,  expanded,  or  use  a  wide  range  of 
language  from  simple  and  casual  to  complex  and  academic.  As  mentioned  in  the  previous 
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section,  we  chose  the  GRCSE  version  of  the  Bloom’s  model  as  a  reference  for  our 
research.  The  reasons  are  three-fold.  First,  this  version  of  Bloom’s  Taxonomy  was  used 
to  classify  the  KSAs  in  the  NPS  SE  Competency  Model.  Using  this  version  of  Bloom’s  to 
classify  the  DAU  curriculum  will  allow  for  a  more  objective  and  direct  comparison. 
Secondly,  the  GRCSE  version  of  Bloom’s  has  the  least  redundant  list  of  verbs  or 
“outcome  descriptors.”  In  some  other  versions  of  Bloom’s,  the  same  verb  is  used  as  an 
outcome  descriptor  in  several  different  Bloom’s  levels.  This  leads  to  ambiguity. 
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Table  2.  Bloom’s  Taxonomy  of  Educational  Outcomes — Cognitive  Domain 

(From  GRCSE,  20 1 1 ,  p.  97-98) 


Level 

Sub- Level 

Competency 

Outcome  Descriptors 

Knowledge 

•  Knowledge  of  specifics 

•  Knowledge  of  terminology 

•  Knowledge  of  specific  facts 

•  Knowledge  of  ways  and 
means  of  dealing  with 
specifics  (processes) 

•  Knowledge  of  the 
universals  and  abstractions 

Ability  to  remember  previously  learned 
material.  Test  observation  and  recall  of 
information;  i.e.,  "bring  to  mind  the 
appropriate  information;"  e.g.,  dates, 
events,  places,  knowledge  of  major  ideas, 
and  mastery  of  subject  matter. 

List,  define,  tell,  describe, 
identify,  show,  label, 
collect,  examine, 
tabulate,  quote,  and 
name  (who,  when,  where, 
etc.). 

Comprehension 

•  Translation 

•  Interpretation 

•  Extrapolation 

Ability  to  understand  information  and 
ability  to  grasp  meaning  of  material 
presented;  e.g.,  translate  knowledge  into 
new  context,  interpret  facts,  compare, 
contrast,  order,  group,  infer  causes, 
predict  consequences,  etc. 

Summarize,  describe, 
interpret,  contrast, 
predict,  associate, 
distinguish,  estimate, 
differentiate,  discuss,  and 

extend. 

Application 

•  Application  of  methods 
and  tools 

•  Use  of  common 
techniques  and  best 
practices 

Ability  to  use  learned  material  in  new  and 
concrete  situations;  e.g.,  use  i nformation, 
methods,  concepts,  and  theories  to  solve 
problems  requiri ng  the  skil Is  or  knowledge 
presented. 

Apply,  demonstrate, 
calculate,  complete, 
illustrate,  show,  solve, 
examine,  modify,  relate, 
change,  classify. 

Analysis 

•  Analysis  of  elements 

•  Analysis  of  relationships 

•  Analysis  of  organizational 
principles 

Ability  to  decompose  learned  material  into 
constituent  parts  in  order  to  understand 
structure  of  the  whole.  This  incl  udes 
seeing  patterns,  organization  of  parts, 
recognition  of  hidden  meanings,  and 
obviously,  identification  of  parts. 

Analyze,  separate,  order, 
explain,  connect,  classify, 
arrange,  divide,  compare, 
select,  explain,  and  infer. 

Synthesis 

•  Production  of  a  unique 

communication 

•  Production  of  a  plan,  or 
proposed  set  of  operations 

•  Derivation  of  a  set  of 

abstract  relations 

Ability  to  put  parts  together  to  form  a  new 
whole.  This  involves  the  use  of  existing 
ideas  to  create  new  ones,  generalizing 
from  facts,  relating  knowledge  from 
several  areas,  and  predicting  and  drawing 
conclusions.  It  may  also  involve  the 
adaptation  of  "general"  solution 
principles  to  the  embodiment  of  a  specific 
problem. 

Combine,  integrate, 
modify,  rearrange, 
substitute,  plan,  create, 
design,  invent,  what-if 
analysis,  compose, 
formulate,  prepare, 
generalize,  and  rewrite. 

Evaluation 

•  Judgements  in  terms  of 
internal  evidence 

•  Judgments  in  terms  of 
external  criteria 

Ability  to  pass  judgment  on  value  of 
material  within  a  given  context  or 
purpose.  This  involves  making 
comparisons  and  discriminating  between 
ideas,  assessing  the  value  of  theories, 
making  choices  based  on  reasoned 
arguments,  verifying  the  value  of  evidence, 
and  recognizi ng  subjectivity. 

award,  choose,  conclude, 
criticize,  decide,  defend, 
determine,  dispute, 
evaluate,  judge,  justify, 
measure,  compare,  mark, 
rate,  recommend,  rule  on, 
select,  agree,  interpret, 
explain,  appraise, 
prioritize,  opinion. 
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Table  3.  Bloom’s  Taxonomy  of  Educational  Outcomes — Affective  Domain 

(From  GRCSE,  201  l,pp.  99-100) 


Level 

Sub-Level 

Competency 

Outcome  Descriptors 

Receiving 

•  Awareness 

•  Willingness  to  receive 

•  Controlled  or  selected 

attention 

The  learner  is  aware  of  stimuli 
and  is  willing  to  attend  to 
them.  The  learner  may  be  able 
to  control  attention  to  the 

stimuli. 

Focuses  on  and  is  aware  of 

aesthetics,  focuses  on  human 
values,  is  alert  to  desirable 
qualities,  and  shows  careful 
attendance  to  input. 

Responding 

•  Acquiescence  in 
responding 

•  Willingness  to  respond 

•  Satisfaction  in  response 

The  learner  makes  a  conscious 
responseto  thestimuli  related 
to  the  aesthetic  or  quality.  At 
this  level  the  learner  expresses 
an  interest  in  the  aesthetic 
things. 

Demonstrates  willing  compliance 
and  obedience  to  regulations  and 
rules,  seeks  broad-based 
information  to  act  upon,  and 
accepts  responsibility  and 
expresses  pleasure  for  own 

situation. 

Valuing 

•  Acceptance  of  a  value 

•  Preference  for  a  value 

•  Commitment 

The  learner  recognizes  worth  in 
the  subject  matter. 

Continuing  desire  to  achieve, 
assumes  responsibility  for,  seeks  to 
form  a  view  on  controversial 

matters,  devotion  to  principles,  and 

faith  in  effectiveness  of  reason. 

Organization 

•  Conceptualizatin  of  a  value 

•  Organizationof  a  value 
system 

The  learner  is  able  to  organize 
a  number  of  values  into  a 
system  of  values  and  can 
determine  the  inter¬ 
relationships  of  the  values. 

Identifies  characteristics  of  an 

aesthetic,  forms  value-based 
judgments,  and  weighs  alternative 
policies. 

Characterization 

•  Generalised  set 

•  Characterization 

The  learner  acts  consistently 
with  the  systems  of  attitudes 
and  values  they  have 
developed.  The  values  and 
views  are  integrated  into  a 
coherent  worldview. 

Readiness  to  revise  judgment  in 
light  of  evidence,  judges  problems 
and  issues  on  their  merit  (not 
recited  positions),  and  develops  a 
consistent  philosophy  of  life. 

Finally,  GRCSE  exists  in  a  systems  engineering  context  and  examples  of  systems 
engineering  outcomes  are  provided,  which  is  helpful  to  our  analysis. 

Bloom’s  Taxonomy  is  a  popular,  tried  and  true  tool  for  categorizing  educational 
outcomes.  The  DAU  used  the  Taxonomy  in  the  creation  of  their  curriculum  and  the  NPS 
SE  Competency  Model  also  used  Bloom’s  levels  as  a  way  of  classifying  KSAs  into 
proficiency  levels.  It  is  a  natural  segue  to  use  Bloom’s  Taxonomy  to  compare  these  two 
models  to  each  other. 
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E.  OTHER  MODELS  AND  SOURCES 

In  addition  to  the  models  and  sources  previously  mentioned  in  this  chapter,  this 
Literature  Review  would  not  be  complete  without  some  reference  to  a  few  institutions 
that  are  pillars  of  the  systems  engineering  world  or  basic  reference  in  a  defense 
acquisition  context.  This  section  will  briefly  review  NDIA,  INCOSE,  GRCSE  and  the 
Defense  Acquisition  Guidebook  and  their  impacts  on  this  research. 

As  mentioned  in  Chapter  I,  an  NDIA  Systems  Engineering  Education  and 
Training  Working  Group  developed  an  original  competency  model  with  29  competencies 
that  was  adopted  by  DAU  and  is  referred  to  in  this  paper  as  the  DAU  SPRDE 
Competency  Model.  The  INCOSE  UK  Competency  Working  Group  is  currently 
considering  revising  this  framework  specifically  for  INCOSE.  The  current  INCOSE 
Competency  Model  is  one  of  the  five  models  that  were  initially  integrated  into  the  NPS 
SE  Competency  Model. 

At  a  6  May  2013  NDIA  Education  and  Training  Committee  meeting,  the 
Competency  Working  Group  defined  its  vision  as  “An  enhanced,  individual  and 
organization-usable  INCOSE  Competencies  Framework  based  on  demand  with 
associated  assessment  instrument.”  Objectives  included  evolving  a  “globally  accepted 
and  marketed  standard  competency  framework,”  that  would 

•  be  tailorable;  useable  by  a  variety  of  different  organizations 

•  be  useful  for  job  candidate  selection 

•  identify  and  mitigate  gaps  in  training 

•  relate  different  competency  models  to  each  other. 

This  project  has  a  great  deal  in  common  with  the  research  being  done  at  NPS  with 
an  important  distinction.  The  INCOSE  Competency  Model  is  geared  towards  systems 
engineering  in  general  while  the  NPS  research  focuses  in  on  naval  acquisition  systems 
engineers.  Eventually,  there  may  be  collaboration,  but  at  this  point,  these  projects  are 
working  independently. 

The  Graduate  Reference  Curriculum  for  Systems  Engineering  (GRCSE)  along 
with  the  Systems  Engineering  Body  of  Knowledge  (SeBOK)  is  part  of  the  BKCASE 
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project  and  was  mentioned  earlier  in  this  chapter  as  a  reference  for  Bloom’s  Taxonomy. 
GRCSE  is  “the  standard  for  what  should  be  in  a  (graduate)  systems  engineering  program 
for  it  to  be  properly  called  systems  engineering”  (Olwell,  2012).  Part  of  the  GRCSE 
framework  was  incorporated  into  the  NPS  SE  Competency  Model  by  use  of  its  version  of 
Bloom’s  Taxonomy.  While  GRCSE  is  a  wonderful  reference  for  the  elements  of  graduate 
systems  engineering  education,  the  DAU  curriculum’s  objectives  are  to  provide  the 
SPRDE  workforce  with  learning  needed  on  the  job,  rather  than  the  curriculum  needed  for 
a  graduate  degree.  The  CNA  SPRDE  Competency  assessment  showed  that  over  half  the 
SPRDE  workforce  had  a  master’s  degree  or  higher.  At  some  point,  GRCSE  could  be  used 
to  compare  the  impact  of  these  graduate  degrees  on  the  SPRDE  workforce,  however,  that 
is  well  beyond  the  scope  of  this  research. 

In  the  review  of  different  sources  of  information  regarding  systems  engineering 
competencies  and  best  practices,  the  Defense  Acquisition  Guidebook  (DAG)  warrants 
mention.  The  DAG  is  particularly  relevant  in  this  case  because  Chapter  4 — Systems 
Engineering  was  just  updated  8  May  2013,  well  beyond  the  halfway  point  of  this 
research.  The  DAG  is  a  product  of  DAU  and  there  was  concern  that  the  CL/POs  and 
ELOs  used  in  our  analysis  had  also  been  revised.  As  of  this  writing,  however,  no  changes 
to  the  curriculum  or  objectives  had  been  posted  on  the  website  or  announced.  The 
revisions  in  Chapter  4  simplified  the  structure  by  reducing  the  number  of  sections  and 
primarily  updated  guidance  to  reflect  current  policy  and  DASD  (SE)  initiatives  (ODASD 
[SE],  2013  May). 

This  section  is  a  bit  of  a  potpourri  of  other  systems  engineering  models  and 
sources  of  information  that  for  one  reason  or  another  had  a  bearing  on  this  research.  The 
inclusion  of  this  infonnation  contributes  to  the  comprehensiveness  and  completes  this 
Literature  Review. 

F.  SUMMARY 

This  chapter  has  reviewed  the  various  tools  and  sources  that  were  used  in  the 
analysis  of  this  research.  Competency  Models  and  Assessments  have  been  discussed  in 
terms  of  characteristics,  uses,  components  and  development.  Two  specific  competency 


38 


models  were  then  focused  on.  Both  the  DAU  SPRDE  Competency  Model  and  the  NPS 
SE  Competency  Model  were  evaluated  in  terms  of  development  and  characteristics  and 
we  also  scrutinized  the  CNA  assessment  of  the  DAU  SPRDE  Competency  Model. 
Bloom’s  taxonomy  was  reviewed  and  determined  to  be  an  appropriate  framework  for  our 
comparative  analysis  and  finally,  we  took  a  brief  look  at  other  systems  engineering 
sources,  models  and  curriculum. 

Now  that  we  have  examined  and  evaluated  the  various  components  of  our 
research,  Chapter  III  will  focus  our  data  and  analysis. 
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III.  DATA  AND  ANALYSIS 


This  chapter  will  examine  the  data  and  methodology  used  to  answer  each  of  the 
primary  and  secondary  research  questions.  The  two  primary  research  questions  involve  a 
comparison  of  the  DAU  SPRDE-SE  Level  III  curriculum  and  the  NPS  SE  Competency 
Model.  This  data  and  analysis  will  be  presented  in  Section  A.  The  two  secondary  research 
questions  require  an  internal  analysis  of  the  DAU  SPRDE-SE  curriculum  to  determine 
how  well  the  course  objectives  are  supported  by  the  learning  objectives  and  how  well  the 
DAU  SPRDE-SE  curriculum  reflects  its  own  competency  model.  The  data  and  analysis 
pertaining  to  the  secondary  research  questions  will  be  presented  in  Section  B.  In  each 
case,  this  research  will  explain  how  the  data  was  obtained  and  its  pertinence  to  this 
research.  The  framework  used  for  comparison  will  be  presented  as  well  as  the  analysis  of 
the  data. 

A.  PRIMARY  RESEARCH:  RELATIONSHIP  BETWEEN  THE  DAU  SPRDE- 

SE  LEVEL  III  CURRICULUM  AND  NPS  SE  COMPETENCY  MODEL 

The  first  two  research  questions  involve  a  comparison  between  the  training 
required  by  DAU  to  gain  SPRDE-SE  Level  III  Certification  and  the  actual  competencies 
needed  to  perfonn  as  a  successful  naval  acquisition  systems  engineer.  The  data  used  to 
represent  each  of  these  entities  will  be  examined  individually  and  then  applied  to  a 
framework  for  comparison. 

The  training  required  by  the  DAU  SPRDE-SE  program  is  identified  as  the 
CL/POs  and  ELOs  of  the  courses  required  for  certification.  The  competencies  necessary 
for  successful  performance  as  a  naval  acquisition  systems  engineer  have  been  identified 
by  the  NPS  SE  Competency  Modeling  Team  as  KSAs.  These  two  models  will  provide 
the  raw  data  for  this  project. 

The  data  used  to  represent  the  DAU  SPRDE-SE  training  requirements  for  Level 
III  Certification  were  obtained  directly  from  the  DAU  iCatalog.  (Defense  Acquisition 
iCatalog,  20 1 3h)  First,  the  courses  required  for  SPRDE-SE  Level  III  Certification  were 
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determined.  The  Certification  requirements  are  hierarchical  and,  therefore,  all  courses 
required  for  Level  I  and  II  are  required  for  SPRDE-SE  Level  III  DAWIA  Certification. 
These  ten  courses  are  shown  in  Figure  7. 

It  is  important  to  note  that  the  data  analyzed  for  this  research  applies  to  the 
training  element  of  DAWIA  Certification.  SPRDE-SE  Certification  also  requires  the 
components  of  education  and  experience.  This  research  focuses  on  the  training  element 
and  does  not  address  the  impacts  of  education  and  experience  on  the  SPRDE-SE 
workforce.  Likewise,  this  research  addresses  only  those  training  courses  identified  by 
DAU  as  part  of  the  “Core  Certification  Standards  required  for  DAWIA  Certification.” 
DAU  also  provides  a  “Core  Plus  Development  Guide”  which  specifies  “Desired  training, 
education  and  experience”  (DAU,  2013).  Courses  in  the  Core  Plus  Development  Guide 
were  not  included  in  this  research’s  analysis. 


DAU  SPRDE-SE  Training  Requirements  for  DAWIA  Certification 

Level  1 

ACQ 101 

Fundamentals  of  Systems  Acquisition 

SYS  101 

Fundamentals  of  Systems  Planning,  Research,  Development, 
and  Engineering 

CLM  017 

Risk  Management 

Level  II 

ACQ  201A 

Intermediate  Systems  Acquisition,  Part  A 

ACQ201B 

Intermediate  Systems  Acquisition,  Part  B 

SYS  202 

Intermediate  Systems  Planning,  Research,  Development,  and 
Engineering,  Part  1 

SYS203 

Intermediate  Systems  Planning,  Research,  Development,  and 
Engineering,  Part  II 

CLE  003 

Technical  Reviews 

Level  III 

SYS  302 

Technical  Leadership  in  Systems  Engineering 

CLL008 

Designing  for  Supportability  in  DoD  Systems 

Figure  7. 

DAU  SPRDE-SE  Training  Requirements 
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Once  the  required  courses  were  identified,  the  objectives  of  these  courses  were 
identified.  The  Course  Learning/  Perfonnance  Objectives  (CL/POs)  and  the  Enabling 
Learning  Objectives  (ELOs)  were  available  from  the  DAU  website  for  the  seven 
acquisition  (ACQ)  and  systems  (SYS)  courses,  but  not  for  the  three  continuing  learning 
(CL)  courses.  Due  to  the  lack  of  availability  of  course  objectives  for  the  CL  courses  and 
the  time  constraints  of  this  research,  the  data  for  the  three  CL  courses  were  not  included 
in  this  analysis.  The  subject  matter  of  these  three  courses  is  risk  management,  technical 
reviews,  and  designing  for  supportability  in  DoD  systems.  While  these  are  relevant 
topics  with  regard  to  systems  engineering  competencies,  the  DAU  estimates  that  these 
courses  take  only  eight,  four,  and  three  hours  respectively  to  complete.  The  other  seven 
courses  are  estimated  to  take  an  average  of  35  hours  each.  (DAU  iCatalog,  20 1 3h)  With 
this  in  mind,  it  was  deduced  that  the  omission  of  these  three  courses  would  not 
substantially  skew  the  results  of  the  analysis.  It  should  be  noted  that  these  CL  courses 
were  included  in  broader  terms  in  the  analysis  of  data  pertaining  to  research  question 
four,  which  will  be  discussed  in  Section  B.  From  the  seven  ACQ  and  SYS  courses,  a  total 
of  654  CL/POs  and  ELOs  were  identified  and  assembled  for  analysis. 

Next,  each  CL/PO  and  ELO  was  categorized  according  to  the  GRCSE  version  of 
Bloom’s  Taxonomy  as  described  in  Chapter  II.  None  of  the  data  fell  into  the  Affective 
Domain;  hence,  all  CL/POs  and  ELOs  were  categorized  into  the  Cognitive  Domain  as 
knowledge,  comprehension,  application,  analysis,  synthesis  or  evaluation.  An  example  of 
the  data  is  presented  Appendix  C. 

With  the  data  for  the  DAU  side  of  the  comparison  established,  the  second  set  of 
data  needed  for  this  comparison  will  be  discussed.  To  represent  the  actual  competencies 
needed  to  perform  as  a  successful  naval  acquisition  systems  engineer,  data  from  the  NPS 
SE  Competency  Model  Version  3  was  used.  The  development  and  pertinence  of  this 
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model  was  discussed  in  detail  in  Chapter  II,  Section  C.  Version  3  of  this  Competency 
Model  consists  of  2,257  KSAs,  which  have  been  mapped  to  the  same  version  of  Bloom’s 
Taxonomy  (GRSCE)  by  the  NPS  SE  Competency  Model  Team.  It  was  recognized  that 
the  assignment  of  Bloom’s  levels  is  somewhat  subjective.  To  reduce  subjectivity,  the 
same  NPS  SME  that  categorized  the  Bloom’s  levels  for  the  NPS  SE  Competency  Model 
KSAs  was  asked  to  review  the  assignment  of  Bloom’s  levels  to  the  DAU  SPRDE-SE 
curriculum.  Revisions  have  been  incorporated  into  the  data. 

With  the  data  from  DAU  and  the  NPS  SE  Competency  Model  identified  and 
mapped  to  Bloom’s  taxonomy,  the  analysis  of  this  data  with  respect  to  the  two  Primary 
Research  Questions  will  be  examined. 

1.  Comparison  of  Cognitive  Levels  of  DAU  SPRDE-SE  Level  III 
Training  Requirements  with  the  NPS  SE  Competency  Model 

Primary  Research  Question  number  1  asks  for  a  comparison  of  the  DAU  SPRDE- 
SE  training  requirements  for  Certification  and  the  competencies  needed  to  perfonn  as  a 
proficient  naval  systems  engineer.  To  answer  this  research  question,  the  cognitive  levels 
of  the  112  DAU  SPRDE-SE  Level  III  CL/POs  and  542  ELOs  were  compared  to  those  of 
the  2,257  KSAs  in  the  NPS  SE  Competency  Model.  Percentages  of  each  cognitive  level 
were  calculated  for  each  DAU  course,  separately  and  collectively,  to  compare  directly 
with  the  percentages  of  each  cognitive  level  found  in  the  NPS  Model.  The  results  are 
shown  in  Figure  8. 
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Figure  8.  Cognitive  Levels  of  DAU  SPRDE-SE  Level  III  Curriculum  and  NPS  SE 

Competency  Model 


2.  Direct  Comparison  between  DAU  SPRDE-SE  CL/POs  and  ELOs  and 
the  NPS  Competency  Model  KSAs 

Primary  Research  Question  number  2  asks  for  a  direct  comparison  of  the  DAU 
SPRDE-SE  Level  III  curriculum  with  the  knowledge,  skills  and  abilities  identified  by  the 
NPS  SE  Competency  Model  to  find  the  overlaps  and  gaps  between  the  required  training 
and  what  is  currently  being  provided  by  DAU.  Because  Version  3  of  the  NPS  model 
absorbed  a  large  percentage  of  the  DAU  curriculum,  as  explained  in  Chapter  II  Section  C, 
this  comparison  is  very  straightforward. 
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Of  the  2,257  KSAs  in  the  NPS  SE  Competency  Model,  613  were  contributed 
directly  from  the  DAU  curriculum.  These  613  newly  defined  KSAs  include  six  duplicates 
and  607  additions  to  version  2  of  the  NPS  Model.  The  other  41  CL/POs  or  ELOs  fall  into 
one  of  the  seven  competencies  that  were  set  aside  from  the  NPS  SE  Competency  Model. 
The  analysis  of  this  data  is  shown  in  Figure  9. 


B.  SECONDARY  RESEARCH:  INTERNAL  DAU  SPRDE-SE  CURRICULUM 
ANALYSIS 

The  secondary  research  questions  involve  an  internal  analysis  of  the  DAU 
SPRDE-SE  curriculum  rather  than  a  comparison  to  another  model.  This  Section  will 
examine  the  data  and  analysis  used  to  answer  the  secondary  research  questions. 

1.  Correlation  between  DAU  SPRDE-SE  CL/POs  and  ELOs 

Secondary  Research  Question  number  1  requires  examination  of  the  DAU 

SPRDE-SE  Enabling  Learning  Objectives  (ELOs)  to  determine  the  extent  to  which  they 
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support  and  reflect  the  Course  Learning/  Performance  Objectives.  While  data  for  the 
primary  research  questions  was  being  analyzed,  it  became  apparent  that  some  ELOs 
mapped  to  a  different  Bloom’s  cognitive  level  than  did  the  overarching  CL/PO.  This 
research  question  addresses  the  extent  to  which  this  inconsistency  exists. 

To  address  this  research  question,  the  Bloom’s  levels  of  ELOs  were  analyzed  with 
respect  to  their  overarching  CL/PO.  First,  the  CL/POs  were  simply  divided  into  two 
categories:  those  with  consistent  ELOs  and  those  with  inconsistent  ELOs.  Next,  the 
CL/POs  with  inconsistent  underlying  ELOs  were  further  subdivided  into  more  specific 
categories.  These  categories  are  based  on  whether  those  inconsistencies  were  result  of 
higher  or  lower  Bloom’s  levels.  The  analysis  shown  in  Figure  10  shows  the  total  number 
and  percentages  of  CL/POs  with  the  following  types  of  inconsistencies: 

•  CL/POs  with  all  lower  cognitive  level  ELOs 

•  CL/POs  with  equal  or  lower  cognitive  level  ELOs 

•  CL/POs  with  equal  or  higher  cognitive  level  ELOs 

•  CL/POs  with  all  higher  cognitive  level  ELOs 

•  CL/POs  with  both  higher  and  lower  cognitive  level  ELOs 

The  impact  of  each  of  these  inconsistencies  will  be  addressed  in  Chapter  VI. 
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Figure  10.  Correlation  between  DAU  SPRDE-SE  CL/POs  and  ELOs 
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2.  Correlation  between  DAU  SPRDE-SE  Level  III  Curriculum  and  the 
DAU  Competency  Model 

Secondary  Research  Question  #2  involves  a  comparison  of  the  DAU  SPRDE-SE 
Level  III  CL/POs  and  the  competency  model  adopted  by  DAU.  Analysis  of  this  data  will 
show  the  extent  to  which  the  courses  required  for  SPRDE-SE  Level  III  Certification 
provide  the  training  needed  by  systems  engineers  as  defined  by  DAUs  own  competency 
model. 

For  this  comparison,  the  original  intent  was  to  map  the  112  CL/POs  to  the  29 
competencies  in  the  DAU  model.  However,  to  achieve  accurate  mapping,  it  was 
necessary  to  also  include  Competencies  30.0 — 35.0  as  identified  by  the  NPS  SE 
Competency  Model.  These  six  competencies  are  not  found  in  the  DAU  Competency 
Model.  The  112  CL/POs  identified  in  the  seven  courses  used  for  prior  analysis  were 
mapped  to  these  35  competencies.  Also  mapped  were  the  three  CL  courses  for  which 
detailed  information  was  not  available.  Each  of  the  three  CL  courses  was  represented  as 
one  CL/PO  for  a  total  of  1 15  CL/POs.  The  analysis  of  this  data  is  shown  in  Figure  11. 
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35.0  Domain  Specialization 
34.0  Facilities  Engineering 
33.0  Contract  Management 
32.0  Financial  Managenemt 
31.0  Individual  and  Interpersonal. 
30.0  Systems  Thinking 
29.0  Professional  Ethics 
28.0  StrategicThinking 
^  27.0  Problem  Solving 
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11.0  Transition 
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9.0  Verification 
8.0  Integration 
7.0  Implementation 
6.0  Architecture  Design 
5.0  Requirements  Analysis 
4.0  Stakeholder  Requirements  Definition 
3.0  Safety  Assurance 
2.0  Modelingand  Simulation 
1.0  Technical  Basisfor  Cost 


Figure  1 1 .  Number  of  DAU  SPRDE-SE  CL/POs  in  each  Systems  Engineering 

Competency 
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C.  SUMMARY 

This  chapter  has  presented  the  data  used  and  the  methodology  applied  to  address 
the  four  objectives  of  this  research.  It  has  also  discussed  the  scope  of  the  data  used  and 
how  that  data  were  developed. 

The  primary  research  questions  involve  data  relating  to  the  DAU  SPRDE-SE 
curriculum  and  the  NPS  SE  Competency  Model.  These  data  were  analyzed  based  on 
cognitive  level  and  by  direct  comparison.  The  secondary  research  questions  focus 
specifically  on  data  reflecting  the  DAU  SPRDE-SE  curriculum  to  provide  an  analysis  of 
how  well  the  DAU  SPRDE-SE  learning  objectives  support  the  course  objectives  and  to 
what  extent  the  SPRDE-SE  curriculum  reflects  the  competencies  identified  in  the  DAU 
SPRDE  Competency  Model. 

Chapter  IV  will  discuss  the  results  of  this  analysis  with  regard  to  each  of  the  four 
research  questions. 
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IV.  FINDINGS/RESULTS 


Chapter  III  provided  the  data  and  analysis  needed  to  answer  each  of  the  four 
research  questions.  This  chapter  will  examine  and  discuss  that  data  and  analysis  to  draw 
conclusions  and  answer  the  four  research  questions.  Findings  that  were  not  obvious  at  the 
start  of  this  research  but  that  emerged  through  the  research  process  will  also  be  discussed. 

A.  PRIMARY  RESEARCH  FINDINGS 

To  answer  the  primary  research  questions,  data  drawn  directly  from  the  DAU 
iCatalog  were  analyzed  and  compared  with  data  provided  by  the  NPS  SE  Competency 
Model.  This  section  will  discuss  that  data  and  their  implications,  draw  conclusions,  and 
provide  answers  to  the  primary  research  questions. 

1.  Findings  Related  to  the  Comparison  of  Cognitive  Levels  of  DAU 
SPRDE-SE  Level  III  Training  Requirements  and  the  NPS  SE 
Competency  Model 

Primary  Research  Question  number  1  asks  to  what  degree  the  DAU  SPRDE-SE 
training  requirements  for  DAWIA  Certification  reflect  the  competencies  needed  to 
perform  as  a  proficient  systems  engineer  in  the  naval  acquisition  enterprise  as  identified 
by  the  NPS  SE  Competency  Model.  Figure  8  shows  a  comparison  of  the  cognitive  levels 
of  these  two  models. 

The  first  subjective  impression  of  Figure  8,  with  regard  to  the  DAU  courses,  is 
that  there  is  a  lot  of  blue,  which  represents  knowledge.  In  fact,  61  percent  of  the  DAU 
SPRDE-SE  CL/POs  and  ELOs  fall  into  the  knowledge  category  and  80  percent  of  all 
DAU  SPRDE-SE  CL/POs  and  ELOs  are  in  the  lower  third  of  the  Bloom’s  Cognitive 
Domain. 

Two  of  the  seven  courses  analyzed  are  particularly  focused  on  knowledge.  ACQ 
101,  the  first  course  in  DAU  SPRDE-SE  Level  I,  focuses  100  percent  on  knowledge, 
which  is  simple  retention.  Because  ACQ  101  is  an  introductory  course,  this  is 
understandable.  What  is  notable,  however,  is  that  98  percent  of  the  objectives  for  SYS 
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202,  a  Level  II  course,  only  require  knowledge.  At  the  second  level  of  a  certification 
program,  higher  cognitive  levels,  such  as  analysis  or  application,  would  be  expected. 

As  for  the  higher  two-thirds  of  the  cognitive  domain,  no  more  than  seven  percent 
of  all  SPRDE-SE  CL/POs  and  ELOs  fall  into  any  one  of  the  four  higher  cognitive  levels. 
Only  20  percent  of  the  entire  DAU  SPRDE  Level  III  training  program  falls  into  the 
higher  two-thirds  of  the  cognitive  domain.  Finally,  none  of  the  SPRDE-SE  course  or 
learning  objectives  applied  to  the  affective  domain  which  reflects  such  skills  as 
engagement,  the  ability  to  lead  systems  projects,  negotiate  outcomes,  communicate  with 
diverse  groups,  and  make  value  judgments.  Mastery  of  these  types  of  abilities  is 
particularly  crucial  for  systems  engineers  and  is  part  of  what  distinguishes  them  from 
other  engineers. 

In  contrast,  it  was  found  that  only  29  percent  of  the  KSAs  identified  in  the  NPS 
SE  Competency  Model  require  the  lower  third  cognitive  levels  of  knowledge  and 
comprehension.  The  need  for  application  abilities  is  strongly  represented  by  26  percent  of 
the  KSAs.  Also  significant  is  that  29  percent  of  all  KSAs  in  the  NPS  SE  Competency 
Model  fall  into  the  affective  domain. 

All  these  comparisons  and  findings  illustrate  differences  between  the  SPRDE-SE 
curriculum  and  the  NPS  SE  Competency  Model  KSAs.  There  are,  however,  similarities 
as  well.  The  data  shows  that  the  highest  three  cognitive  levels  (analysis,  synthesis  and 
evaluation)  are  similarly  represented  in  both  models.  These  three  highest  cognitive  levels 
account  for  14  and  16  percent  of  the  SPRDE-SE  learning  objectives  and  NPS  KSAs 
respectively. 

These  findings  show  that  the  majority  of  the  learning  objectives  of  the  SPRDE-SE 
Level  III  curriculum  require  substantially  lower  cognitive  levels  of  thinking  than  do  the 
actual  competencies  required  to  perform  as  a  proficient  systems  engineer  in  the  naval 
acquisition  enterprise  as  identified  by  the  NPS  SE  Competency  Model.  The  SPRDE-SE 
curriculum  specifically  shows  a  gap  or  lack  of  development  of  skills  pertaining  to  the 
application  of  knowledge  and  as  well  as  skills  falling  into  the  affective  domain. 
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The  data  also  shows  an  overlap  between  the  models.  The  highest  three  cognitive 
levels  of  analysis,  synthesis  and  evaluation  are  similarly  required  in  both  the  SPRDE-SE 
curriculum  and  the  NPS  SE  Competency  Model.  These  highest  three  cognitive  levels  are 
represented  with  14  and  16  percent,  respectively,  of  the  two  models. 

2.  Findings  related  to  the  Direct  Comparison  between  DAU  SPRDE-SE 
CL/POs  and  ELOs  and  the  NPS  Competency  Model  KSAs 

Primary  Research  Question  number  2  asks  for  the  similarities,  differences,  gaps 
and  overlaps  between  DAU  SPRDE-SE  curriculum  and  NPS  SE  Competency  Model 
KSAs.  Some  important  similarities,  differences,  gaps  and  overlaps  in  cognitive  levels 
were  identified  in  the  findings  developed  to  answer  the  first  research  question.  Further 
analysis  was  done,  however,  to  directly  compare  the  CL/POs  and  ELOs  from  the 
SPRDE-SE  curriculum  to  the  KSAs  in  the  NPS  SE  Competency  Model.  This  comparison 
will  show  the  difference  between  the  KSAs  needed  by  a  proficient  naval  acquisition 
systems  engineer  and  the  training  provided  by  DAU. 

The  data,  illustrated  in  Figure  9,  shows  a  total  of  2,257  KSAs  in  the  NPS  SE 
Competency  Model,  v.3.  Of  these,  613,  or  only  27  percent  of  the  NPS  Model,  are 
addressed  by  the  DAU  curriculum.  Conversely,  these  613  KSAs  represent  94  percent  of 
the  DAU  SPRDE-SE  curriculum.  The  other  41  CL/POs  and  ELOs,  or  6  percent  of  the 
DAU  SPRDE-SE  curriculum,  applies  to  either  soft  skills  or  Domain  Specialization  and 
were  not  included  in  the  current  version  of  the  NPS  SE  Competency  Model.  Recall  that 
professional  skills  are  recognized  as  being  important  for  systems  engineers  and  that  these 
competencies  will  likely  be  incorporated  in  the  NPS  SE  Competency  Model  or  a  sister 
model,  as  it  continues  to  develop.  This  will  result  in  the  inclusion  of  an  even  higher 
percentage  of  the  DAU  SPRDE-SE  curriculum. 

These  findings  show  that  a  27  percent  overlap  and  a  73  percent  gap  exists  in  the 
training  of  the  knowledge,  skills  and  abilities  needed  to  perform  as  a  proficient  naval 
acquisition  systems  engineer.  The  implications  are  both  good  and  bad.  The  good  news  is 
that  the  DAU  is  providing  27  percent  of  what  systems  engineers  need  and  that  is  a  good 


53 


start.  The  challenge  will  be  in  determining  where  naval  acquisition  systems  engineers  are 
getting  the  other  73  percent  of  the  knowledge,  skills  and  abilities  they  need. 

This  section  has  determined  the  extent  to  which  the  DAU  SPRDE-SE  training 
requirements  reflect  the  competencies  needed  to  perform  as  a  proficient  systems  engineer 
in  the  naval  acquisition  enterprise.  Similarities,  differences,  gaps  and  overlaps  between 
DAU  SPRDE-SE  training  curriculum  and  NPS  SE  Competency  Model  KSAs  have  been 
identified  and  analyzed.  Next,  findings  pertaining  to  the  secondary  research  questions 
will  be  discussed. 

B.  SECONDARY  RESEARCH  FINDINGS 

The  secondary  research  questions  were  developed  to  provide  insight  into  the 
strength  and  consistency  of  the  DAU  training  curriculum  and  to  determine  how  well  it 
reflects  the  DAU  SPRDE  Competency  Model.  This  analysis  is  in  contrast  to  the  primary 
research  objectives  which  compared  the  SPRDE-SE  curriculum  to  the  NPS  SE 
Competency  Model.  The  findings  related  to  this  internal  analysis  of  the  DAU  SPRDE 
curriculum  and  competencies  will  be  presented  in  this  section. 

1.  Findings  related  to  the  Correlation  between  DAU  SPRDE-SE  CL/POs 
and  ELOs 

Secondary  Research  Question  #1  asks  for  the  extent  to  which  the  DAU  SPRDE- 
SE  Enabling  Learning  Objectives  support  and  relate  to  the  overarching  Course  Learning/ 
Performance  Objectives.  These  findings  will  help  to  determine  whether  the  actual 
objectives  of  the  DAU  SPRDE-SE  training  are  being  provided  for  through  the  curriculum 
and  are  illustrated  in  Figure  10. 

The  initial  data  analysis  showed  that  59  percent  of  CL/POs  were  supported  by 
ELOs  with  inconsistent  cognitive  levels.  This  is  one  reason  why  CL/POs  and  ELOs  were 
given  equal  weight  as  KSAs  in  the  NPS  SE  Competency  Model.  This  finding,  in  and  of 
itself,  is  not  enough  to  determine  impact  on  the  curriculum.  The  specific  characteristics  of 
the  inconsistencies  were  analyzed  and  these  have  important  implications.  This  analysis 
further  divided  the  66  inconsistently  supported  CL/POs  into  five  categories  of  various 
types  of  inconsistency. 
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The  first  category  of  inconsistency  examined  is  CL/POs  completely  supported  by 
ELOs  that  required  lower  cognitive  levels.  This  category  is  the  most  pertinent  to  this 
research  and  represents  12  percent  of  all  SPRDE-SE  CL/POs.  A  specific  example  can  be 
seen  in  the  data  for  ACQ  201B,  CL/PO  3.0  shown  in  Appendix  C.  In  these  cases,  all  the 
underlying  ELOs  require  lower  cognitive  levels,  such  as  the  ability  to  identify  or  select 
information,  than  those  of  the  overarching  CL/PO,  which  may  require  synthesis  or 
evaluation.  This  category  is  particularly  significant  in  that  it  is  difficult  to  understand  how 
the  student  is  expected  to  progress  from  the  lower  cognitive  level  ELOs  to  achieve  the 
higher  cognitive  level  CL/POs. 

The  other  four  categories  all  include  ELOs  with  cognitive  levels  that  are  equal  to 
or  higher  than  the  overarching  CL/PO.  While  these  inconsistencies  may  warrant  further 
research,  the  inconsistencies  themselves  do  not  present  issues.  Because  the  cognitive 
levels  are  hierarchical,  it  is  easy  to  understand  how  a  student  who  has  achieved  a  higher 
cognitive  level  ELO  could  then  apply  that  to  achieve  a  lower  cognitive  level  CL/PO. 
Likewise,  ELOs  requiring  cognitive  levels  lower  than  and  equal  to  the  CL/PO  could  be 
explained  as  developmental. 

These  findings  conclude  that  59  percent  of  the  overarching  CL/POs  are  comprised 
of  inconsistent  ELOs  and  that  at  least  12  percent  of  the  CL/POs  are  not  sufficiently 
supported  by  the  underlying  ELOs.  It  would  follow  that  further  research  into  the 
effectiveness  and  validity  of  the  DAU  SPRDE-SE  curriculum  and  whether  goals  and 
objectives  are  being  met  may  be  warranted. 

2.  Findings  related  to  the  Correlation  between  DAU  SPRDE-SE  CL/POs 
and  the  DAU  SPRDE  Competency  Model 

Secondary  Research  Question  number  2  focuses  on  the  extent  to  which  the  DAU 
SPRDE-SE  Course  Learning/  Performance  Objectives  reflect  the  DAU  Competency 
Model.  Recall  that  the  three  CL  courses  not  included  in  the  data  for  the  primary  research 
were,  in  fact,  included  in  this  analysis,  each  categorized  as  one  CL/PO.  This  inclusion  has 
resulted  in  a  more  comprehensive  set  of  data  for  this  research  question. 
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When  the  115  CL/POs  were  mapped  to  the  29  competencies  represented  in  the 
SPRDE  Competency  Model,  various  interesting  results  emerged.  This  section  will 
examine  the  degree  to  which  there  is  a  link  between  the  DAU  SPRDE-SE  curriculum  and 
the  associated  DAU  SPRDE  Competency  Model.  A  strong  link  suggests  continuity  and 
strength  in  the  development  of  the  SPRDE  program  whereas  a  weak  link  implies  that  the 
curriculum  does  not  support  the  identified  competencies.  The  analysis  of  this  data  is 
shown  in  Figure  11.  These  findings  will  be  discussed  individually  and  a  resultant 
comprehensive  conclusion  will  be  drawn. 

First,  5  of  the  29  competencies  had  no  associated  CL/POs.  That  means  that  DAU 
provides  no  training  for  17  percent  of  the  competencies  identified  in  the  DAU  SPRDE 
Competency  Model.  These  competencies  are  Safety  Assurance,  System  Assurance, 
System  of  Systems,  Problem  Solving,  and  Strategic  Thinking.  This  does  not  show 
evidence  of  a  strong  link  between  the  curriculum  and  the  competency  model.  To  take 
this  point  one  step  further,  15  competencies,  or  over  half,  were  supported  by  zero  or  one 
CL/PO.  This  would  strongly  demonstrate  that  the  competencies  in  the  model  are  not 
being  addressed  by  the  DAU  SPRDE  training  curriculum. 

Secondly,  18  CL/POs,  or  15  percent  of  all  CL/POs  in  the  DAU  SPRDE-SE 
curriculum,  mapped  to  six  competencies  not  identified  in  the  DAU  SPRDE  Competency 
Model.  Again,  this  does  not  suggest  a  strong  link  between  the  SPRDE-SE  curriculum  and 
the  SPRDE  Competency  Model.  These  six  competencies  are  Financial  Management, 
Contract  Management,  Facilities  Engineering,  Individual  and  Interpersonal  Skills, 
Systems  Thinking,  and  Domain  Specialization.  With  the  exception  of  Domain 
Specialization  and  Individual  and  Interpersonal  Skills  (a  soft  skill),  it  has  been 
determined  that  the  other  four  competencies  are  important  for  naval  acquisition  systems 
engineers.  This  is  evidenced  by  their  inclusion  in  the  NPS  SE  Competency  Model.  The 
implication  of  this  finding  is  that  these  competencies  should  be  added  to  the  DAU 
Competency  Model  as  well. 

Next,  Competency  23.0  Acquisition,  accounted  for  the  highest  number  of  CL/POs 

being  supported  by  39,  or  34  percent,  of  all  SPRDE-SE  CLC/POs.  This  finding  shows 

that  the  DAU  SPRDE-SE  curriculum  is  focused  on  the  DoD  acquisition  environment  as 
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opposed  to  a  more  generic  systems  engineering  setting.  This  concentration,  however, 
could  be  to  the  detriment  of  training  for  other  important  competencies  and  does  not 
necessarily  demonstrate  a  strong  link  between  the  DAU  SPRDE-SE  curriculum  and  the 
SPRDE  Competency  Model.  An  associated  finding  is  that  this  strength  in  acquisition 
training  added  viability  to  the  NPS  SE  Competency  Model  when  the  DAU  SPRDE-SE 
curriculum  was  incorporated  in  its  third  version. 

The  competencies  with  the  next  highest  number  of  supporting  CL/POs  are;  Risk 
Management  with  10,  Contract  Management  with  nine,  and  Technical  Planning  with 
seven.  All  other  competencies  had  no  more  than  four  supporting  CL/POs. 

As  for  the  rest  of  the  competencies  in  the  model,  23  competencies  were  each 
addressed  by  one  to  10  CL/POs.  These  23  competencies  accounted  for  58,  or  50  percent, 
of  all  SPRDE-SE  CL/POs.  Of  these  23  competencies,  13  were  supported  by  only  one 
CL/PO. 

To  summarize,  of  the  115  SPRDE-SE  CL/POs: 

•  50  percent  applied  to  23  of  the  29  competencies  identified  in  the  SPRDE- 
SE  Competency  Model. 

•  34  percent  applied  to  one  competency 

•  18  percent  applied  to  six  competencies  outside  the  SPRDE  Model 

•  0  percent  applied  to  five  competencies 

While  these  findings  are  all  interesting,  only  the  last  one  directly  relates  to  the 
secondary  research  question.  The  fact  that  five  of  the  29  competencies  are  not  addressed 
by  any  of  the  CL/POs  conversely  means  that  the  DAU  SPRDE-SE  CL/POs  only  reflect 
83  percent  of  the  competencies  in  their  own  model. 

This  section  has  presented  the  findings  related  to  the  secondary  research 
questions.  Specifically,  it  has  determined  that  12  percent  of  the  DAU  Course  Learning  / 
Performance  Objectives  are  not  supported  cognitively  by  the  associated  Enabling 
Learning  Objectives  and  that  the  DAU  curriculum  only  reflects  83  percent  of  the 
competencies  in  the  DAU  SPRDE-SE  Competency  Model.  Next,  findings  other  than 
those  related  to  the  primary  and  secondary  research  questions  will  be  discussed. 
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C.  OTHER  FINDINGS 

In  the  course  of  the  exploring  the  background  of  this  research,  developing  the  data 
and  analysis;  and  discovering  the  findings,  two  other  related  findings  have  emerged. 

First,  the  SPRDE-SE  career  field  is  extremely  broad.  As  referenced  in  Chapter  I, 
28.1  percent  of  the  entire  DAW  is  classified  as  SPRDE.  Of  that,  92  percent  is  SPRDE- 
SE.  This  career  field  currently  includes  computer  engineers,  structural  engineers, 
electrical  engineers,  mechanical  designers,  software  engineers,  human  factors  engineers, 
reliability  engineers  and  others.  All  of  these  professionals  are  not  systems  engineers.  The 
data  would  suggest  that  this  career  field  requires  finer  definition. 

Another  interesting  finding  is  that  contracting  is  a  vital  competency  for  SEs.  This 
competency  is  part  of  what  differentiates  the  DAU  and  NPS  competency  models  from 
other,  more  mainstream  SE  models  such  as  INCOSE.  It  was  found  that  contracting  is  the 
third  most  highly  addressed  competency  in  the  DAU  SPRDE-SE  curriculum,  accounting 
for  8  percent  of  all  CL/POs.  Only  acquisition  and  risk  management  were  addressed  with  a 
more  concentrated  level  of  curriculum.  The  third  version  of  the  NPS  SE  Competency 
Model  has  incorporated  contracting  as  one  of  the  28  competencies.  Because  the  DAU 
SPRDE-SE  curriculum  reflects  such  a  high  percentage  of  contracting  course  objectives,  it 
would  seem  plausible  that  this  competency  be  added  to  their  model  as  well. 

D.  SUMMARY 

This  chapter  has  discussed  the  findings  resulting  from  the  previously  presented 
data  with  regard  to  the  four  research  questions.  Two  other  findings  that  emerged  in  the 
course  of  this  research  were  also  discussed. 

It  was  found  that  the  DAU  SPRDE-SE  curriculum  requires  a  much  lower 
cognitive  level  that  the  competencies  actually  required  to  perform  as  a  proficient  naval 
acquisition  systems  engineer  and  addresses  no  thought  processes  in  the  affective  domain. 
The  research  also  discovered  that  the  DAU  SPRDE-SE  curriculum  covers  27  percent  of 
the  required  KSAs.  Another  finding  is  that  less  than  half  of  the  DAU  SPRDE-SE 
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curriculum  CL/POs  are  consistently  supported  by  their  underlying  ELOs.  Finally,  the 
DAU  SPRDE-SE  curriculum  does  not  accurately  reflect  the  DAU  SPRDE  Competency 
Model. 

The  next  chapter  will  draw  conclusions  from  these  findings,  summarize  this 
research  and  provide  recommendations  for  further  study. 
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V.  CONCLUSIONS,  RECOMMENDATIONS,  SUMMARY  AND 
AREAS  FOR  FURTHER  RESEARCH 

A.  CONCLUSIONS  AND  RECOMEND ATIONS 

The  findings  of  this  research  lead  to  several  conclusions  about  the  DAU  SPRDE- 
SE  training  program,  required  curriculum  and  competency  model. 

First,  this  research  has  established  that  a  gap  exists  in  the  training  for  naval 
acquisition  systems  engineers  with  relation  to  cognitive  abilities  and  educational 
outcomes.  Naval  acquisition  systems  engineers  need  training  requiring  higher  cognitive 
levels  than  the  training  provided  by  the  DAU  SPRDE-SE  certification  program.  A  full  80 
percent  of  the  of  the  DAU  SPRDE-SE  certification  curriculum  requires  only  the  lower 
cognitive  levels  of  knowledge  and  comprehension.  Specifically  lacking  is  training 
pertaining  to  the  application  of  knowledge  as  well  as  training  with  respect  to  those 
abilities  developed  by  the  affective  domain.  Affective  outcomes,  such  as  developing 
consensus,  communication  across  professional  cultures,  and  exercising  judgment  are 
particularly  essential  for  systems  engineers.  These  higher  levels  of  educational  outcomes 
are  not  addressed  sufficiently  by  the  DAU  SPRDE-SE  certification  curriculum.  To 
provide  comprehensive  training  for  DON  systems  engineers,  a  detennination  must  be 
made  as  to  where  and  how  these  higher  levels  of  thinking  will  be  developed. 

Secondly,  DAU  has  developed  a  SPRDE-SE  training  curriculum  to  satisfy 
DAWIA  requirements.  This  training  does  not  define  or  describe  what  naval  acquisition 
systems  engineers  are  or  what  they  need  to  know  to  excel.  The  DAU  SPRDE-SE  training 
curriculum  required  for  DAWIA  certification  addresses  only  27  percent  of  the 
knowledge,  skills  and  abilities  required  by  naval  acquisition  systems  engineers.  While 
this  classifies  as  a  component  of  comprehensive  training,  it  also  highlights  the  need  to 
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identify  the  source  for  the  other  73  percent.  While  DAU  is  providing  the  AWF  with 
DAWIA  certification,  this  training  is  not  comprehensively  sufficient  to  address 
proficiency  achievement  for  naval  acquisition  systems  engineers. 

Thirdly,  more  research  is  needed  to  determine  whether  the  information  taught  in 
the  DAU  SPRDE-SE  curriculum  reflects  the  desired  outcomes.  This  research  discovered 
that  almost  60  percent  of  the  overarching  course  objectives  were  supported  with 
inconsistent  learning  objectives.  This  could  be  due  to  an  incomplete  listing  or 
misinterpretation  of  the  learning  objectives.  However,  this  research  concludes  that 
confirmation  is  needed  that  the  27  percent  of  needed  training  referenced  in  the  preceding 
paragraph  is  actually  being  achieved  and  not  just  wishful  thinking. 

Another  conclusion  drawn  from  this  research  is  that  the  DAU  SPRDE-SE 
certification  curriculum  is  not  aligned  with  the  DAU  SPRDE  Competency  Model.  The 
training  curriculum  fails  to  address  five  of  the  29  competencies  in  any  way,  however  it 
does  address  six  competencies  not  present  in  the  SPRDE  Model.  Most  of  these 
competencies  have  been  identified  by  the  NPS  SE  Competency  Model  as  being  valid  for 
inclusion.  In  conclusion,  the  link  between  the  DAU  SPRDE-SE  training  curriculum  and 
the  DAU  SPRDE  Competency  Model  is  weak.  Closer  alignment  of  the  DAU  curriculum 
and  SE  competencies  could  result  in  a  stronger  overall  training  program. 

Finally,  to  properly  focus  training  on  this  highly  specialized  workforce,  the 
systems  engineering  career  field  needs  to  be  more  accurately  defined.  Currently,  the 
SPRDE-SE  career  field  is  too  broad  and  includes  many  other  types  of  systems  engineers. 
Systems  engineering  heavily  influences  the  outcomes  of  acquisition  programs.  Attention 
to  the  accurate  training  of  this  workforce  is  imperative  to  the  success  of  major  acquisition 
programs. 
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B.  SUMMARY 

This  research  provides  an  analysis  of  the  training  requirements  and  competencies 
for  naval  acquisition  systems  engineers.  The  DAU  SPRDE-SE  training  curriculum 
required  for  DAWIA  certification  has  been  compared  and  contrasted  in  different  ways, 
both  externally  with  the  NPS  SE  Competency  Model,  and  internally  with  DAU’s  own 
competency  model.  The  strength  of  the  curriculum  itself  has  also  been  examined. 

Although  the  DAU  SPRDE  certifications  represent  the  assimilation  of  some 
systems  engineering  capabilities,  there  is  a  large  gap  between  the  training  required  for 
SPRDE  certification  and  the  competencies  required  to  perform  successfully  as  a  naval 
systems  engineer.  This  research  has  also  shown  that  the  DAU  SPRDE-SE  curriculum 
does  not  accurately  reflect  the  29  competencies  identified  in  the  DAU  SPRDE 
Competency  Model. 

C.  AREAS  FOR  FURTHER  RESEARCH 

The  development  of  this  research  has  brought  to  light  several  new  questions  and 
areas  for  further  investigation. 

The  conclusions  suggest  that  further  investigation  is  needed  into  the  SPRDE-SE 
certification  curriculum  to  determine  whether  the  information  taught  in  the  DAU 
SPRDE-SE  courses  actually  reflects  the  desired  outcomes.  Also,  a  closer  look  into  the 
DAU  Competency  Model  is  warranted  to  see  what  affects  it  does  or  doesn’t  have  on  the 
SPRDE-SE  learning  objectives. 

The  NPS  SE  Competency  Model  is  also  fertile  ground  for  further  research.  This 
model  is  in  its  third  iteration  and  no  doubt  further  versions  will  be  developed.  Intention 
exists  to  incorporate  the  professional  competencies  that  were  set  aside  in  the  second 
version.  It  has  been  demonstrated  that  these  competencies,  while  not  unique  to  systems 
engineers,  are  crucial  for  systems  engineers.  These  competencies  could  be  recognized  as 
a  secondary  competency  model  or  another  completely  separate  competency  model  that 
would  apply  to  a  wider  range  of  career  fields. 


63 


The  NPS  SE  Competency  Model  is  developing  as  a  universal  competency  model 
that  can  be  tailored  or  customized  for  different  organizations/  agencies.  One  associated 
body  of  research  is  currently  being  conducted  at  SPAWAR  using  the  NPS  SE  model  as  a 
framework.  As  this  use  of  the  NPS  SE  Competency  model  is  developed,  its  applications 
will  grow  exponentially.  This  application  could  be  particularly  useful  for  agencies  outside 
the  DoD,  such  as  the  U.S.  Coast  Guard  (DHS),  who  are  not  under  the  umbrella  of  DAU 
and  require  the  development  of  their  own  training  programs. 

Finally,  this  research  has  established  that  a  gap  exists  in  the  training  needed  by 
naval  acquisition  systems  engineers  for  their  development.  This  leads  directly  to  the 
question  of  where  and  how  the  SPRDE  workforce  can  obtain  the  training  needed  to  fill 
this  gap  in  knowledge,  skills  and  abilities  not  addressed  in  the  DAU  SPRDE  training 
program.  While  this  research  has  focused  on  training,  competencies  are  also  developed 
through  formal  education  and  work  experience.  The  impact  of  education  and  work 
experience  on  the  gap  in  training  identified  by  this  research  is  highly  relevant  for  further 
study. 
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APPENDIX  A.  DAU  SPRDE  COMPETENCY  MODEL 


Table  4.  SPRDE-SE/PSE  Competency  Model  (After  Lasley-Hunter,  2011,  pp.  83-87) 


Units  of 

Competence 

Competencies 

Competency  Elements 

Analytical 

Competency  1.  Technical  Basis  for 
Cost 

Element  1.  Provide  technical  basis  for  comprehensive 
cost  estimates  and  program  budgets  that  reflect 
program  phase  requirements  and  best  practices  using 
knowledge  of  cost  drivers,  risk  factors,  and  historical 
documentation  (e.g.  hardware,  operational  software, 
lab/support  software). 

Competency  2.  Modeling  and 
Simulation 

Element  2.  Develop,  use,  and/or  interpret  modeling  or 
simulation  in  support  of  systems  acquisition 
throughout  the  entire  Defense  Acquisition 

Management  System  (changed  from  framework). 
Understand  and  use  M&S  from  other  domains  in 
support  of  systems  acquisition. 

Competency  3.  Safety  Assurance 
(changed  from  Safety  Analysis) 

Element  3.  Review  Safety  Assurance  artifacts  to 
if  requirements  and  constraints  needed  to  meet  SE 
design  goals  for:  Safe  For  Intended  Use  (SFIU),  war¬ 
fighter  survivability,  user  safety,  software  safety, 
environmental  safety,  Programmatic  Environmental, 
Safety  and  Health  Evaluations  (PESHE),  and/or 
Critical  Safety  applications  were  met. 

Competency  4.  Stakeholder 
Requirements  Definition 
(Requirements  Development) 

Element  4.  Elicit  inputs  from  relevant  stakeholders 
translate  the  inputs  into  technical  requirements. 

Competency  5.  Requirements 
(Logical  Analysis) 

Element  5.  Define  and  refine  system,  subsystem, 
lower-level  functional  and  performance  requirements 
and  interfaces  to  facilitate  the  Architecture  Design 
proccess. 

Competency  6.  Architecture  Design 
(Design  Solution) 

Element  6. Identify  the  full  set  of  decomposed 
and  overall  design  considerations,  across  the  full 
system  life-cycle  and  in  all  operating  environments, 
that  should  be  addressed  during  systems  engineering 
in  order  to  obtain  the  “'best  value  "for  the  user. 

Element  7.  Track  and  manage  design  considerations 
(boundaries,  interfaces,  standards,  available 

production  process  capabilities,  performance  and 
behavior  characteristics)  to  ensure  they  are  properly 
addressed  in  the  technical  baselines. 

Element  8.  Translate  outputs  from  the  Stakeholder 
Requirements  Definition  and  Requirements  Analysis 
processes  to  generate  a  final  design  or  physical 
architecture,  using  the  Architecture  Design  process, 
including  reviews  of  alternative  designs. 

Element  9.  Conduct  walkthroughs  with  stakeholders 
ensure  that  requirements  will  be  met  and  will  deliver 
planned  systems  results  under  all  combination  of 
design 
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Units  of 

Competence 

Competencies 

Competency  Elements 

Analytical 

Competency?.  Implementation 

Element  10.  Fabricate  hardware/code  software  to 
realize  system  elements. 

Competency  8.  Integration 

Element  11.  Use  the  Integration  process  to  ensure 
the  lower-level  system  elements  are  incorporated  into 
higher-level  system  elements  and  that  the  final  system 
is  incorporated  into  its  operational  environment. 

Competency  9.  Verification 

Element  12.  Design  and  implement  a  testing 
process  to  compare  a  system  against  required  system 
capabilities,  to  link  Modeling  and  Simulation  (M&S), 
Developmental  Test  and  Evaluation  (DT&E)  and 
Operational  Test  and  Evaluation  (OT&E)  together,  in 
order  to  document  system  capabilities,  limitations,  and 
risks. 

Element  13.  Assess  the  "Did  you  build  it  right?" 
question  question  in  the  Verification  Process  by 
determining  if  system  elements  meet  design 
specifications  as  defined  in  the  functional,  allocated, 
and  product  baselines  using  reviews  of  test  plans, 
among  other  inputs. 

Competency  10.  Validation 

Element  14.  Assess  the  "Did  you  build  the  right 
thing?"question  in  the  Validation  Process,  by 
determining  whether  there  was  a  satisfactory 
performance  of  systems  within  their  intended 
operational  environment,  using  validation  metrics. 

Competency  11.  Transition 

Element  15.  Move  the  system  elements  to  the  next 
Level  in  the  physical  architecture  or  move  the  end 
item  to  the  user  for  use. 

Competency  12.  System 

Assurance 

Element  16.  Apply  and  execute  the  appropriate 
systems  engineering,  software  assurance,  and 
certification  related  policies,  principles,  and  practices 
across  all  levels  and  phases  of  an  acquisition  program 
to  increase  the  level  of  confidence  that  a  system 
functions  as  intended,  is  free  from  exploitable 
vulnerabilities,  and  protects  critical  program 
information. 

Competency  13.  Reliability, 
Availability  &  Maintainability  (RAM) 

Element  17.  Determine  how  best  to  integrate  and 
phase  RAM  into  the  systems  engineering 
processes  across  the  design. 

Element  18.  Evaluate  the  RAM  of  systems,  including 
the  following:  Maintenance  Engineering/  Sustaining 
Engineering  review  and  assessment;  considerations 
of  different  use  environments,  operators,  and 
maintainers;  and  the  monitoring  of  performance 
versus  predictions  of  performance. 
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Units  of 

Competence 

Competencies 

Competency  Elements 

Technical 

Management 

Competency  14.  Decision  Analysis 

Element  19.  Develop  and/or  use  a  Decision 

Analysis  process  to  incorporate  factors  such  as 
operational  environment,  mission  performance,  cost, 
and  design  consid-  erations  into  decision-making. 

Competency  15.  Technical 

Planning 

Element  20.  Use  Technical  Planning  to  ensure 
technical  and  technical  management  processes, 
technical  reviews,  and  the  program  office’s 
organization  are  documented  in  the  Systems 
Engineering  Plan  and  applied  throughout  a  system’s 
acquisition  lifecycle. 

Competency  16.  Technical 
Assessment 

Element  21.  Develop  and/or  use  Technical 
Assessment  metrics  (i.e.,  Technical  Performance 
Measures,  Measures  of  Effectiveness,  requirements 
compliance,  and  risk  assessments)  to  measure 
technical  progress,  review  life-cycle  costs,  and  assess 
the  effectiveness  of  plans  and  requirements. 

Competency  17.Configuration 
Management 

Element  22.  Use  a  Configuration  Management 
process  to  track  configuration  changes  and  ensure 
that  a  product  or  system’s  attributes  are  consistent 
with  its  requirements,  evolving  technical  baseline  over 
its  life-cycle. 

Competency  18.  Requirements 
Management 

Element  23.  Use  Requirements  Management  to 
trace  back  to  user-defined  capabilities  and  other 
sources  of  requirements,  and  to  document  all 
changes  and  the  rationale  for  those  changes. 

Competency  19.  Risk  Management 

Element  24.  Develop  a  Risk  Management  Plan  to 
cover  system  and  software  risk  elements  in  order  to 
assess  and  manage  the  risks  throughout  the  life-cycle. 

Element  25.  Apply  the  Risk  Management  process 
across  an  acquisition  program  to  manage  program 
technical  risk  accounting  for  all  relevant  design 
considerations. 

Competency20.  Technical  Data 
Management 

Element  26.  Evaluate  how  the  Technical  Data 
Management  planning  process  is  applied  in  the  context 
of  the  Acquisition  Strategy  to  acquire,  access, 
manage,  protect,  and  apply  technical  data  during  the 
system  life-cycle. 

Competency  21.  Interface 
Management 

Element  27.  Apply  the  Interface  Management 
process  to  ensure  interface  compatibility  both  within 
the  system,  including  software  systems,  and  with 
external  systems. 

Element  28.  Evaluate  how  Interface  Management 
techniques  ensure  that  all  internal  and  external 
interface  changes  in  requirements  are  properly 
documented  and  communicated  in  accordance  with 
the  configuration  management  plan. 
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Units  of 

Competence 

Competencies 

Competency  Elements 

Technical 

Management 

Competency  22.  Software 
Engineering 

Element  29.  Software  Measures.  Use  quantitative 
methods  to  assess  and  track  software  progress 
against  a  baseline  (planned  vs.  actual)  and  provide 
status  updates  in  order  to  make  timely  program 
decisions. 

Element  30.  Integration  of  Software  and  Systems 
Engineering.  Integrate  essential  acquisition  and 
sustainment  activities  related  to  software  through  the 
use  of  multidisciplinary  teams  to  optimize  design, 
manufacturing,  and  supportability  processes. 

Element  31.  Software  Impact  on  Acquisition 

Strategy.  Determine  software-related  considerations 
and  risks  that  must  be  addressed  as  part  of  the 
system  acquisition  strategy. 

Element  32.  Software  Requirements.  Evaluate 
inputs  from  relevant  stakeholders  that  translate  into 
functional  and  technical  requirements  that  are 
documented,  managed,  traceable  and  verifiable 
through  the  software  lifecycle  process  and  describe 
the  desired  behavior  of  the  software  system  to  be 
built  to  satisfy  the  intended  users. 

Element  33.  Software  Architecture.  Understand  the 
software  structure  of  the  system,  including  the 
definition  of  software  components,  and  the 
relationships  between  software  components,  the 
system,  and  the  operational  architectures. 

Competency  23.  Acquisition 

Element  34.  Determine  the  appropriate  amount  of 
systems  engineering  and  the  resources  needed  during 
each  acquisition  phase,  across  the  full  acquisition  and 
system  lifecycle,  to  achieve  acceptable  levels  of  risk 
for  entry  into  the  next  acquisition  phase. 

Element  35.  Assess  the  proposed  solution’s 
operational  viability  and  costs  of  alternative  systems 
during  the  Materiel  Solution  Analysis  (formerly  called 
Concept  Refinement)  Phase,  taking  into  account  the 
design  considerations  to  achieve  a  balanced  system 
design. 

Element  36.  Integrate  proven  technologies,  develop 
new  hardware/software  prototypes,  evaluate  solutions, 
and  determine  performance  requirements  during  the 
Technology  Development  Phase  to  ensure  that  the 
cost,  schedule,  and  other  constraints  are  met  and  that 
risks  are  reduced. 
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Units  of 

Competence 

Competencies 

Competency  Elements 

Technical 

Management 

Competency  23.  Acquisition 

Element  37.  Integrate  hardware,  software,  and 
human  systems,  protect  critical  program  information, 
ensure  safety  and  affordability,  and  reduce 
manufacturing  risks  during  the  Engineering  and 
Manufacturing  Development  (formerly  called  System 
Development  and  Demonstration)  Phase  to 
demonstrate  supportability  and  interoperability  within 
incremental  stages  of  system  development. 

Element  38.  Apply  a  Low-Rate  Initial  Production 
approach  to  attain  Initial  Operational  Capability  and  Full 
Rate  Production  and  Deployment,  considering 
Diminishing  Manufacturing  Sources  and  Material 
Shortages  (DMSMS);  assess  changes  in  the  design 
of  manufactur-  ing  processes,  and  apply  continuous 
testing  and  evalua-  tion  practices  during  the  Production 
and  Deployment  Phase  to  reveal  manufacturing  and 
production  problems  and  ensure  continuous 
enhancements  to  the  product. 

Element  39.  Plan  the  Logistics  Management  system 
manpower  needs  and  support  plans,  and  apply  within 
a  Performance-Based  Logistics  (PBL)  environment, 
for  the  full  system  lifecycle,  to  ensure  effective  use  of 
the  system. 

Technical 

Management 

Competency  24.  Systems 
Leadership 

Element  40.  Lead  teams  by  providing  proactive  and 
technical  direction  and  motivation  to  ensure  the  proper 
application  of  systems  engineering  processes  and 
the  overall  success  of  the  technical  management 
process. 

Competency  25.  System  of 

Systems 

Element  41.  Translate  the  unique  operational, 

and  architecture  considerations  of  System  of  Systems 

programs  into  the  systems  engineering  approach. 

Professional 

Competency  26.  Communication 

Element  42.  Communicate  technical  and  complex 
in  a  clear  and  organized  manner,  both  verballyand  in 
writing,  to  inform  and  persuade  others  to  adopt  and  act 
on  specific  ideas. 

Competency  27.  Problem  Solving 

Element  43.  Make  recommendations  using  technical 
knowledge  and  experience,  developing  a  clear 
understanding  of  the  system,  identifying  and  analyzing 
problems  using  a  Total  Systems  approach,  weighing 
the  relevance  and  accuracy  of  information,  accounting 
for  interdependencies,  and  evaluating  alternative 
solutions. 

Competency  28.  Strategic  Thinking 

Element  44.  Formulate  and  ensure  the  fulfillment  of 
priorities,  and  plans  consistent  with  the  long  term 
business  and  competitive  interests  of  the  organization 
in  a  global  environment. 

Competency  29.  Professional 

Ethics 

Element  45.  Maintain  strict  compliance  to 
and  standards  of  conduct  in  engineering  and 
business  practices  to  ensure  integrity  across  the 
acquisition  lifecycle. 
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APPENDIX  B.  NPS  SE  COMPETENCY  SAMPLE 


Table  5.  Competency:  1.0  TECHNICAL  BASIS  FOR  COST 
Career  Level:  SE-1  Entry  Level 


Knowledge,  Skills  &  Abilities  (KSA's) 

Cognitive  &  Affective  Skill  Levels 

Education 
&  Training 

On  the  Job 
Experience 

Professional 

Development 

Total 

Knowledge,  Comprehension, 

Receiving  Phenomena  &  Valuing 

Contribute  to  the  recording  of  project 
budget  activities  in  accounting  and 
financial  systems 

X 

1 

Contribute  to  timely  and  accurate  full 
cost  budget  information  (such  as  labor, 
procurement,  travel  estimates)  to  project 
managers  when  requested 

X 

1 

Describe,  identify  or  define  general 
principles  of  full  cost  and  Earned  Value 
Management  (EVM)  and  their  application 
in  the  project  environment 

X 

1 

Describe,  identify  or  define  processes 
for  cost  estimating  technical  work 
products 

X 

1 

Understand  budgeting  process  and 
accounting  and  financial  management 
techniques 

X 

1 

Understand  the  project  budget 
development  process 

X 

1 

Application,  Analysis,  Responding  to 
Phenomena  &  Organization 

Perform  cost  estimating  on  technical 
work  products 

X 

1 

Use  Work  Breakdown  Structure  (WBS) 
as  a  tool  for  tracking  actual  vs.  estimated 
costs  and  use  this  information  to  revise 
cost  models  appropriately 

X 

1 

Synthesis,  Evaluation  &  Internalizing 
Values 

1 

Identify  significant  resource  needs  and 
issues  for  the  system  of  interest 

X 

1 

Grand  Total 

9 
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Table  6.  Competency:  1 .0  TECHNICAL  BASIS  FOR  COST 
Career  Level:  SE-2  Journey  Level 


Knowledge  Skills  &  Abilities  (KSA's) 

Cognitive  &  Affective  Skill  Levels 

Education 
&  Training 

On  the  Job 
Experience 

Professional 

Development 

Total 

Knowledge,  Comprehension,  Receiving 
Phenomena  &  Valuing 

1 

Contribute  timely  and  accurate  data  (such 
as  budget  estimates)  to  project  managers  to 
the  project  budget  development  process 

X 

1 

Application,  Analysis,  Responding  to 
Phenomena  &  Organization 

6 

■ 

Ensure  that  a  cost  analysis  requirements 
description  (CARD)  is  developed  and 
maintained 

X 

Ensure  that  all  project  needs  are 
adequately  covered  and  properly  time 
phased  in  the  budget  submission  for  projects 
of  low  to  medium  complexity 

X 

1 

Ensure  that  the  cost  estimate  covers  the 
entire  project  life  cycle 

X 

1 

Ensure  use  of  advanced  models  and 
techniques  for  cost  estimating  during 
relevant  project  life  cycle  phases 

X 

1 

Ensure  use  of  straightforward  and  well- 
documented  models  and  techniques  for  cost 
estimating  during  relevant  project  life  cycle 
phases 

X 

1 

Negotiate  budgets  and  contracts  with  line 
organizations  or  contractors 

X 

1 

Synthesis,  Evaluation  &  Internalizing 
Values 

Evaluate  resource  management  products 
and  understand  their  implications  for  the 
system  of  interest 

X 

1 

Prepare  a  project-operating  plan  that 
projects  the  cost  required  to  proceed 
according  to  the  Project  Management  Plan 
(PMP) 

X 

1 

Review  and  approve  cost  estimates  for 
subsystem  elements 

X 

1 

Grand  Total 

10 
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Table  7.  Competency:  1 .0  TECHNICAL  BASIS  FOR  COST 
Career  Level:  SE-3  Expert  Level 


Knowledge  Skills  &  Abilities  (KSA's) 

Cognitive  &  Affective  Skill  Levels 


Knowledge,  Comprehension, 

Receiving  Phenomena  &  Valuing 


Education  &  On  the  Job 
Training  Experience 


Professional 

Development 


Describe,  identify  or  define  processes 
and  techniques  for  working  with 
stakeholders  to  effectively  deal  with  a 
dynamic  budget  environment 


Synthesis,  Evaluation 

Internalizing  Values 


& 


Evaluate  and  reconcile  independent 
cost  estimates  with  advocacy  cost 
estimates 


X 


Grand  Total 
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APPENDIX  C.  DAU  SPRDE-SE  LEVEL  III  COURSE  DATA 


Table  8.  ACQ  20 IB  Data 


COURSE 

UALfjB 

BLOOM'S 

CATEGORY 

DAU 

COMPETENCY 

ACQ201B 

1.0 

Determine  how  IPTleadership  concepts  can  be  used  to  overcome 
barriers  to  effective  teamwork,  based  on  real  world  experience. 

comprehension 

24.0  Systems 

Engineering 

Leadership 

ACQ201B 

1.1 

Relate  keytenets  of  IPPDto  planningand  executingan  acquisition 

program. 

knowledge 

24.0  Systems 

Engineering 

Leadership 

ACQ201B 

1.2 

Identify  the  aids  and  barriers  to  successful  IPTimplementation. 

knowledge 

24.0  Systems 

Engineering 

Leadership 

ACQ201B 

1.3 

Identify  the  Supervisory,  Participative  and  Team  leadership  styles. 

knowledge 

24.0  Systems 

Engineering 

Leadership 

ACQ201B 

1.4 

Describe  how  different  leadership  styles  impact  the  effectiveness  ofan 

IPT. 

comprehension 

24.0  Systems 

Engineering 

Leadership 

ACQ201B 

1.5 

Identify  the  behaviors  and  characteristics  of  effective  teams. 

knowledge 

24.0  Systems 

Engineering 

Leadership 

ACQ201B 

2.0 

Resolve  an  acquisition-related  dilemma  by  prioritizing  ethical  values  and 
considering  how  choices  impact  the  welfare  of  others. 

application 

29.0 

Professional 

Ethics 

ACQ201B 

2.1 

Identifythe  characteristics  ofa  "successful"  defense  acquisition  program 
from  a  variety  of  perspectives. 

knowledge 

29.0 

Professional 

Ethics 

ACQ201B 

2.2 

Identify  core  ethical  values  critical  to  decision  making  in  the  acquisition 

environment. 

knowledge 

29.0 

Professional 

Ethics 

ACQ201B 

2.3 

Identifythe  steps  ofthe  Principled  Decision  Making  Model. 

knowledge 

29.0 

Professional 

Ethics 

ACQ201B 

2.4 

Resolve  an  ethical  dilemma  by  applying  the  steps  ofthe  Principled 

Decision  Making  .Model. 

application 

29.0 

Professional 

Ethics 

ACQ201B 

3.0 

Evaluate  alternative  approaches  to  meet  a  needed  capability  based  on 
affordability,  schedule  and  technical  considerations. 

evaluation 

14.0  Decision 

Analysis 

ACQ201B 

3.1 

Given  a  user's  requirement  and  selected  concept,  select  an  appropriate 
approach  from  the  perspective  ofthe  system  developer,  to  meet  the 
requirement. 

knowledge 

14.0  Decision 

Analysis 

ACQ201B 

3.2 

Identifythe  three  major  dimensions  of  program  risk  used  to  analyze 
technical  approaches  duringthe  Materiel  Solution  Analysis  Phase  (cost, 
schedule  and  performance). 

knowledge 

19.0  Risk 

Management 

ACQ201B 

3.3 

Identifythe  concept  of  Cost  as  an  Independent  Variable  (CAIV)  in  relation 
to  an  acquisition  program. 

knowledge 

1.0  Technical 

Basis  for  Cost 

ACQ201B 

3.4 

Relate  the  concepts  of  affordability  and  Cost  as  an  Independent  Variable 
(CAIV)tothe  planningofan  acquisition  program. 

knowledge 

1.0  Technical 

Basis  for  Cost 

ACQ201B 

3.5 

Working  in  a  student-led  IPT,  demonstrate  the  behaviors  and 

characteristics  ofan  effective  team. 

application 

24.0  Systems 

Engineering 

Leadership 
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ACQ201B 

4.0 

Prepare  an  acquisition  strategy  program  structure  chart  showing 
appropriate  interrelationship(s)  of  the  various  business  and  technical 
functions  involved  in  planning  and  executing  the  program. 

analysis 

15.0  Technical 

Planning 

ACQ201B 

4.01 

Given  an  acquisition  program  scenario  with  information  on  technology 
maturity,  fundingand  JCIDS  documentation,  identifythe  correct  starting 
point  for  the  program  in  the  acquisition  lifecycle. 

analysis 

15.0  Technical  1 
Planning 

ACQ201B 

4.02 

Identifythe  correct  type  appropriated  funds  needed  by  phase  and  work 
effort. 

comprehension 

15.0  Technical  1 
Planning 

ACQ201B 

4.03 

Given  an  acquisition  program  structure  chart  identifythe  correct 
sequence  and  timing  of  technical  reviews  by  phase  and  work  effort. 

analysis 

15.0  Technical 

Planning 

ACQ201B 

4.04 

Given  an  acquisition  program  structure  chart  identifythe  correct 
sequence  and  timing  of  developmental  and  operational  test  events  by 
phase  and  work  effort. 

analysis 

15.0  Technical  1 
Planning 

ACQ201B 

4.05 

Given  an  acquisition  program  structure  chart  identifythe  correct 
sequence  and  timing  of  lifecycle  logistics  planningand  execution  efforts 
by  phase  and  work  effort. 

analysis 

15.0  Technical  1 
Planning 

ACQ201B 

4.06 

Given  an  acquisition  program  structure  chart,  identifythe  appropriate 
contract  types  by  phase  and  work  effort. 

analysis 

15.0  Technical 

Planning 

ACQ201B 

4.07 

Given  an  acquisition  program  structure  chart,  identifythe  timing  of  major 
hardware  deliverables  by  phase  and  workeffort. 

analysis 

15.0  Technical 
Planning 

ACQ201B 

4.08 

Relate  the  capability  documents  (ICD,CDD,CPD)tothe  correct  phases  of 
the  acquisition  system. 

knowledge 

23.0 

Acquisition 

ACQ201B 

4.09 

Identifythe  evolutionary  acquisition  strategy  approach. 

knowledge 

23.0 

Acquisition 

ACQ201B 

4.10 

Identifythe  single  step  acquisition  strategy  approach. 

knowledge 

23.0 

Acquisition 

ACQ201B 

5.0 

Modify,  present,  and  defend  an  acquisition  strategy  to  accommodate  a 
change  in  program  funding  levels. 

synthesis 

23.0 

Acquisition 

ACQ201B 

5.01 

Identifythe  proper  response  to  a  program  fundingcut. 

comprehension 

23.0 

Acquisition 

ACQ201B 

5.02 

Given  a  program  fundingcut  identifythe  potential  impacts  on  industry. 

comprehension 

23.0 

Acquisition 

ACQ201B 

6.0 

Develop  portions  ofa  source  selection  plan,  includingsource  selection 

criteria. 

synthesis 

Contract 

Management 

ACQ201B 

6.01 

Identify  howthe  Government  communicates  performance  requirements 
in  solicitations. 

knowledge 

Contract 

Management 

ACQ201B 

6.02 

Identifythe  role  ofvarious  IPT members  in  developingthe  solicitation. 

knowledge 

Contract 

Management 

ACQ201B 

6.03 

Identifythe  purpose  ofevaluation  criteria  and  howthe  criteria  are 
developed. 

knowledge 

Contract 

Management 

ACQ201B 

6.04 

Develop  evaluation  criteria  in  a  source  selection. 

synthesis 

Contract 

Management 

ACQ201B 

7.0 

Apply  the  iterative  SEsteps  to  develop  outputs  ofthe  systems 
engineering  process  in  orderto  verify  they  meet  a  given  requirement. 

application 

23.0 

Acquisition 

ACQ201B 

7.1 

Given  a  summary  Capability  Development  Document  (CDD)  and  a  system 
concept,  determine  whetherthe  concept  addresses  all  user 
requirements. 

application 

5.0 

Requirements 

Analysis 

ACQ201B 

7.2 

Identifythe  overall  purpose  ofthe  systems  engineering  process. 

knowledge 

30.0  Systems  1 
Thinking 

ACQ201B 

7.3 

Identifythe  technical  processes  that  make  up  the  overall  systems 
engineering  process. 

knowledge 

23.0 

Acquisition 

ACQ201B 

7.4 

Identifythe  technical  management  processes  used  to  control  and 
manage  the  overall  systems  engineering  process. 

knowledge 

23.0 

Acquisition 

ACQ201B 

7.5 

Identifythe  main  inputs  and  outputs  ofthe  overall  systems  engineering 
process. 

knowledge 

23.0 

Acquisition 

ACQ201B 

7.6 

Given  an  acquisition  scenario  within  an  IPT  environment,  develop  and 
present  selected  outputs  ofthe  systems  engineering  process  steps. 

synthesis 

23.0 

Acquisition 
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ACQ201B 

8.0 

Given  a  program  schedule,  explain  the  role  oftest  and  evaluation  (DT&E, 
OT&E,  LFT&E)  in  the  systems  engineeringand  acquisition  management 

processes. 

comprehension 

9.0 

Verification 

ACQ201B 

8.1 

Identifythe  characteristics  and  purposes  of  Developmental  Test  and 
Evaluation  (DT&E). 

knowledge 

9.0 

Verification 

ACQ201B 

8.2 

Identifythe  characteristics  and  purposes  of  Operational  Test  and 
Evaluation  (OT&E). 

knowledge 

9.0 

Verification 

ACQ201B 

8.3 

Identifythe  characteristics  and  purposes  of  Live  Fire  Test  and  Evaluation 
(LFT&E). 

knowledge 

9.0 

Verification 

ACQ201B 

8.4 

Given  a  test  event  description,  correctly  identifythe  type  of  testing  being 
accomplished. 

comprehension 

9.0 

Verification 

ACQ201B 

8.5 

Given  a  program  schedule,  correctly  identify  opportunities  for  combined 
DT/OT. 

comprehension 

9.0 

Verification 

ACQ201B 

8.6 

Identifythe  risks  and  benefits  associated  with  combining  DT and  OT 

events. 

knowledge 

9.0 

Verification 

ACQ201B 

9.0 

Analyze  actual  verses  planned  technical  performance  data  in  risk  areas 
to  indicate  potential  problems  that  may  prevent  a  system  from  being 
operationally  effective  and  suitable. 

analysis 

19.0  Risk 

Management 

ACQ201B 

9.1 

Identify  potential  riskareas  based  on  technical  performance  data. 

analysis 

19.0  Risk 

Management 

ACQ201B 

9.2 

Identifythe  role  oftechnical  performance  measures  in  the  systems 
engineering  process. 

knowledge 

16.0  Technical 

Assessment 

ACQ201B 

10.0 

Given  a  segment  of  contract  workand  associated  tasks,  plan  and 
schedule  the  tasks  and  resources  necessary  to  complete  contract  work 
within  cost  and  schedule  constraints. 

synthesis 

Contract 

Management 

ACQ201B 

10.1 

Apply  the  fully  burdened  rate  to  labor  hours  to  correctly  calculate 

contractor's  costs. 

application 

Contract 

Management 

ACQ201B 

10.2 

Distinguish  correctly  between  direct  and  indirect  costs  on  a  contract. 

comprehension 

Contract 

Management 

ACQ201B 

10.3 

Given  a  simple  Gantt  chart  with  defined  task  relationships,  identifythe 
critical  path. 

comprehension 

15.0  Technical  1 
Planning 

ACQ201B 

10.4 

Given  a  completed  Gantt  chart  with  the  critical  path  identified,  identify 
cost  and  schedule  risks  in  the  plan. 

comprehension 

15.0  Technical  1 
Planning 

ACQ201B 

10.5 

Given  a  completed  Gantt  chart  with  the  critical  path  identified,  explain 
cost  and  schedule  risks  in  the  plan. 

comprehension 

15.0  Technical 

Planning 

ACQ201B 

11.0 

Select  a  best  value  contractor  by  comparing  contractor  proposals  and 

test  results  to  source  selection  criteria. 

analysis 

Contract 

Management 

ACQ201B 

11.1 

Apply  evaluation  criteria  in  a  source  selection. 

application 

Contract 

Management 

ACQ201B 

11.2 

Identifythe  best  value  approach  to  source  selection. 

knowledge 

Contract 

Management 

ACQ201B 

11.3 

Applya  selected  quantitative  tool  (e.g.  decision  matrix)to  resolve  a 
problem. 

application 

14.0  Decision 

Analysis 

ACQ201B 

12.0 

The  student  will  be  able  to  analyze  and  contractor  performance 
indicators  to  identify  trends  and  problems. 

analysis 

Contract 

Management 

ACQ201B 

12.01 

Given  earned  value  data  calculate  cost  variance,  schedule  variance,  cost 
performance  indexand  schedule  performance  index. 

application 

Contract 

Management 

ACQ201B 

12.02 

Given  cost  variance,  schedule  variance,  SPI  &  CPI  explain  the  program's 
cost  and  schedule  status. 

comprehension 

Contract 

Management 

ACQ201B 

12.03 

Given  the  Actual  Cost,  Target  Cost,  Target  Profit,  Target  Price,  Share  Line, 
and  Ceiling  Price  on  a  Fixed  Price  Incentive  Firm  Target  Contract,  correctly 

calculate  the  Final  Contract  Price. 

application 

Contract 

Management 

ACQ201B 

13.0 

Given  a  scenario,  apply  key  software  acquisition  management  principles 
needed  to  make  sound  decisions  for  planning  a ndexecuting  an 
acquisition  program. 

application 

22.0  Software  1 
Engineering 

ACQ201B 

13.01 

Identify  common  ways  that  software-intensive  projects  have  gotten  into 
trouble. 

knowledge 

22.0  Software  1 
Engineering 

ACQ201B 

13.02 

Identify  "Best  Practices"  that  may  be  appropriate  for  the  acquisition  of 
software-intensive  systems. 

comprehension 

22.0  Software  1 
Engineering 
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ACQ201B 

13.03 

Identify  the  aspects  ofthe  Net  Ready  KPP  as  it  applies  to  acquisition  of 
Information  Technology  (e.g.  interoperability,  architecture,  information 
assurance). 

comprehension 

22.0  Software 

Engineering 

ACQ201B 

13.04 

Identify  the  benefits  and  risks  associated  with  using  Commercial  Off  The 
Shelf  (COTS)  .software 

knowledge 

22.0  Software 

Engineering 

ACQ201B 

13.05 

Explain  the  relationship  between  software  development  activities  and 
the  systems  engineering  process. 

comprehension 

22.0  Software 

Engineering 

ACQ201B 

13.06 

Explain  the  impact  of  a  new  requirement  on  various  functional  areas. 

comprehension 

22.0  Software 

Engineering 

ACQ201B 

13.07 

Identifythe  impacts  of  a  new  program  requirement  on  the  following 
functional  areas:  Program  Management,  Systems  Engineering, 
Contracting,  Lifecycle  Logistics,  Financial  Management,  Software 
Acquisition  Management,  &  Test  and  Evaluation. 

comprehension 

22.0  Software 

Engineering 

ACQ201B 

14.00 

Analyze  a  reliability  problem  from  multiple  perspectives  and  select  and 

defend  a  solution. 

evaluation 

13.0 

Reliability, 

Availability, 

and 

Maintainabilit 

y 

ACQ201B 

14.01 

Explain  the  interrelationship  between  selected  functional  areas  (e.g., 
contracting,  finance,  systems  engineering) and  acquisition  logistics. 

comprehension 

13.0 

Reliability, 

Availability, 

and 

Maintainabilit 

y 

ACQ201B 

14.02 

Explain  why  it  is  important  to  influence  system  design  for  supportability. 

comprehension 

13.0 

Reliability, 

Availability, 

and 

Maintainabilit 

y 

ACQ201B 

14.03 

Explain  the  relationship  of  Reliability,  Availability,  and  Maintainability 
(RAM)to  Acquisition  Logistics,  and  its  impact  on  system  performance, 
operational  effectiveness  (includingsupport),  logistics  planning,  and  life- 
cycle  cost. 

comprehension 

13.0 

Reliability, 

Availability, 

and 

Maintainabilit 

y 

ACQ201B 

14.04 

Identifythe  impacts  of  a  supportability  problem  on  the  following 
functional  areas:  Program  Management,  Systems  Engineering, 
Contracting,  Lifecycle  Logistics,  Financial  Management,  Quality 

Assurance  &  Manufacturing,  &  Test  and  Evaluation. 

application 

13.0 

Reliability, 

Availability, 

and 

Maintainabilit 

y 

ACQ201B 

14.05 

Explain  how  instability  of  requirements,  design,  and  production 
processes  impact  program  cost  and  schedule. 

comprehension 

13.0 

Reliability, 

Availability, 

and 

Maintainabilit 

y 

ACQ201B 

15.00 

Given  a  scenario,  identifythe  major  contract  administration  activities. 

comprehension 

Contract 

Management 

ACQ201B 

15.01 

Explain  the  interrelationship  between  selected  functional  areas  (e.g.,  life 
cycle  logistics,  finance,  systems  engineering)  and  contracting. 

comprehension 

Contract 

Management 

ACQ201B 

15.02 

Identifythe  causes  and  consequences  of  constructive  changes. 

knowledge 

Contract 

Management 

ACQ201B 

15.03 

Identifythe  complementary  roles  and  responsibilities  ofthe  contracting 
officer  and  the  program  manager  in  their  partnership  throughout  the 
acquisition  process. 

knowledge 

Contract 

Management 

ACQ201B 

16.00 

Given  a  scenario,  apply  the  procedures,  rules  and  public  laws  associated 
with  the  execution  of  DoD  budgets. 

analysis 

23.0 

Acquisition 

ACQ201B 

16.01 

Identifythe  public  laws  (i.e..  Misappropriation  Act,  Anti-deficiency  Act, 
Bona  Fide  Need  Rule)  that  apply  to  the  use  of  appropriated  funds  in  DoD 
acquisition. 

comprehension 

23.0 

Acquisition 

ACQ201B 

16.02 

Select  the  appropriate  public  law  (i.e.,  Misappropriation  Act,  Anti¬ 
deficiency  Act,  Bona  Fide  Need  Rule)that  a^^ies  to  the  use  of 
appropriated  funds  under  specific  circumstances. 

comprehension 

23.0 

Acquisition 

IACQ201B 

17.00 

Analyze  the  elements  of  manufacturingas  they  relate  to  a  systems 
performance  problem  usinga  qualitative  tool  (cause  and  effect/fishbone 
diagram). 

analysis 

14.0  Decision 

Analysis 

ACQ201B 

17.01 

Identify  the  elements  of  manufacturing  (5Ms). 

knowledge 

6.0 

Architecture 

Design 

ACQ201B 

17.02 

Explainthe  considerations/concerns  ofthe  elements  of  manufacturing 
(5 Ms)  and  how  other  areas  are  affected. 

comprehension 

6.0 

Architecture 

Design 

ACQ201B 

17.03 

Explain  the  impact  of  manufacturingon  cost,  schedule  and  performance. 

comprehension 

6.0 

Architecture 

Design 

ACQ201B 

17.04 

Explain  the  use  ofthe  5  Whys  root  cause  determination  method  in 
identifying  potential  root  causes. 

comprehension 

14.0  Decision 

Analysis 

ACQ201B 

17.05 

Explain  the  multi-votingtechnique  to  narrow  large  lists  of  possibilities 
into  smaller,  more  manageable,  lists. 

comprehension 

14.0  Decision 

Analysis 

ACQ201B 

18.00 

Analyze  the  impact  of  supportability  issues  on  system 
readiness/performance  and  other  functional  areas.  E.g.  contracts, 
finance,  systems  engineeringand  acquisition  logistics. 

analysis 

23.0 

Acquisition 

ACQ201B 

18.01 

Synthesize  several  approaches  to  solving  a  program  supportability  issue 
(obsolescence). 

synthesis 

23.0 

Acquisition 

ACQ201B 

18.02 

Evaluate  approaches  to  solvinga  program  supportability  issue 
(obsolescence). 

evaluation 

23.0 

Acquisition 

ACQ201B 

18.03 

Recommend  the  best  to  solvinga  program  supportability  issue 
(obsolescence). 

evaluation 

23.0 

Acquisition 

ACQ201B 

18.04 

Identify  the  proper  DoD  Appropriation  Category  to  be  used  to  budget  for 
each  ofthe  three  phases  of  a  Product  Improvement  Program. 

comprehension 

23.0 

Acquisition 

ACQ201B 

18.05 

Assess  the  impact  ofthe  failure  to  execute  funds  in  accordance  with 
program  plans. 

knowledge 

23.0 

Acquisition 

ACQ201B 

18.06 

Recognize  how  configuration  management  impacts  all  functional 
disciplines  (e.g.,  test,  logistics,  manufacturing,  etc.). 

comprehension 

17.0 

Configuration 

ACQ201B 

18.07 

Demonstrate  the  interrelationship  between  selected  functional  areas, 
e.g.,  contracting,  finance,  systems  engineering,  and  life  cycle  logistics. 

comprehension 

23.0 

Acquisition 

ACQ201B 

18.08 

Identify  tools/best  practices/techniques  available  in  the  systems 
engineering  process  to  achieve  the  principal  goals  of  supportability 
analyses. 

knowledge 

23.0 

Acquisition 

ACQ201B 

18.09 

Apply  performance  based  metrics  to  a  program  supportability  problem 
(e.g.  obsolescence). 

application 

23.0 

Acquisition 

ACQ201B 

18.10 

Apply  performance  or  outcome  based  logistics  principles  to  solving  a 
program  obsolescence  issue. 

application 

23.0 

Acquisition 
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